Super-resolution method, terminal equipment and storage medium

Abstract

The invention discloses a super-resolution method, terminal equipment and a storage medium. The method comprises the following steps: constructing a mapping relation between super-resolution pixels and reference pixels according to super-resolution multiples; when super-resolution processing is carried out on the original image, determining a pixel value of a reference pixel corresponding to a current pixel in the original image based on a mapping relation between the super-resolution pixel and the reference pixel; and determining the pixel value of the current pixel according to the pixel value of the reference pixel, and determining a super-resolution image corresponding to the original image based on the pixel value of the current pixel.

Description

Technical Field

[0001] The present invention relates to the field of super-resolution technology, and in particular to a super-resolution method, terminal equipment, and storage medium. Background Art

[0002] Super-resolution (SR), abbreviated as super-resolution, commonly used super-resolution video solutions include methods based on traditional computer vision (CV) algorithms and artificial intelligence (AI) neural networks, both of which involve image scaling calculations.

[0003] However, the coordinate value calculation in the common resizing method is highly complex and involves a large amount of repeated calculations, which increases unnecessary time consumption, reduces the processing efficiency of video super-resolution, and further affects the super-resolution performance of the terminal device. Summary of the invention

[0004] The embodiments of the present application provide a super-resolution method, a terminal device, and a storage medium, which can reduce the complexity and repeatability of calculations in the super-resolution process, improve the processing efficiency of video super-resolution, and thereby improve the super-resolution performance of the terminal device.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a super-resolution method, the method comprising:

[0007] Construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple;

[0008] When super-resolution processing is performed on the original image, based on the mapping relationship between the super-resolution pixels and the reference pixels, a pixel value of a reference pixel corresponding to a current pixel in the original image is determined;

[0009] The pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel.

[0010] In a second aspect, an embodiment of the present application provides a terminal device, the terminal device comprising: a construction unit, a determination unit,

[0011] The construction unit is used to construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple;

[0012] The determination unit, when performing super-resolution processing on the original image, determines the pixel value of the reference pixel corresponding to the current pixel in the original image based on the mapping relationship between the super-resolution pixel and the reference pixel; determines the pixel value of the current pixel according to the pixel value of the reference pixel, and determines the super-resolution image corresponding to the original image based on the pixel value of the current pixel.

[0013] In a third aspect, an embodiment of the present application provides a terminal device, wherein the terminal device includes a processor and a memory; wherein:

[0014] The memory is used to store a computer program that can be run on the processor;

[0015] The processor is used to execute the method described in the first aspect above when running the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the method described in the first aspect above is implemented.

[0017] The embodiment of the present application provides a super-resolution method, a terminal device, and a storage medium, wherein the terminal device constructs a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple; when super-resolution processing is performed on the original image, the pixel value of the reference pixel corresponding to the current pixel is determined in the original image based on the mapping relationship between the super-resolution pixels and the reference pixels; the pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel. That is to say, in the embodiment of the present application, the terminal device can perform coordinate value calculation according to the super-resolution multiple in advance before starting to perform super-resolution processing on the original image, and complete the construction of the mapping relationship between the super-resolution pixels and the reference pixels, so that in the process of super-resolution processing, the mapping relationship between the super-resolution pixels and the reference pixels that has been constructed can be directly used to complete the calculation of the pixel value, so that the repeated and highly complex coordinate value calculation processing can be avoided. It can be seen that in the present application, the coordinate value calculation and pixel value calculation processes in the resize calculation are separated, and the mapping relationship between the super-resolution pixels and the reference pixels is determined in advance, thereby reducing the complexity and repeatability of the calculation in the super-resolution process, improving the processing efficiency of the video super-resolution, and thus improving the super-resolution performance of the terminal device. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of the implementation flow of the super-resolution method proposed in the embodiment of the present application;

[0019] Figure 2 is a schematic diagram of the mapping relationship between super-resolution pixels and reference pixels;

[0020] Figure 3 Schematic diagram of replacing pointers for integers;

[0021] Figure 4 A schematic diagram of vectorized calculation;

[0022] Figure 5 Schematic diagram of the pixel value calculation process Figure 1 ;

[0023] Figure 6 This is a schematic diagram of the coordinate value calculation process;

[0024] Figure 7 Schematic diagram of the pixel value calculation process Figure 2 ;

[0025] Figure 8 Schematic diagram of the structure of the terminal device proposed in the embodiment of the present application Figure 1 ;

[0026] Fig. 9 Schematic diagram of the structure of the terminal device proposed in the embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for ease of description, only the parts related to the related applications are shown in the drawings.

[0028] Super-resolution (SR), abbreviated as super-resolution, refers to the process of using optics and related optical knowledge to restore image details and other data information based on known image information. Simply put, it increases the resolution of the image to prevent its image quality from deteriorating.

[0029] Commonly used super-resolution video solutions include methods based on traditional computer vision (CV) algorithms and artificial intelligence (AI) neural networks, both of which involve image scaling calculations.

[0030] Among them, resize calculation is used for image scaling processing and is a common pixel operation algorithm. The resize calculation mainly includes two steps: coordinate value calculation and pixel value calculation. The coordinate value calculation process is more complicated and time-consuming, and the calculation process for different frames is the same, which is a repeated calculation.

[0031] For example, the resizing calculation involved in the video super-resolution algorithm can be roughly divided into two steps: coordinate value calculation and pixel value calculation. First, the coordinates of the pixel points in the input image on which the output image depends are calculated, taking the image size before and after resizing as a parameter. i ×H i , output W o ×H o For the image with input coordinates (I x , I y ) pixel point, the output coordinates (O x , O y ) is calculated as:

[0032] O x =I x ×W o / W i (1)

[0033] O y =I y ×H o / H i (2)

[0034] For bilinear interpolation, one output point depends on four input points, and the coordinates of the other three input points are (I x , I y +1), (I x +1,I y ), (I x +1,I y +1). At the same time, two weight values, w1 and w2, need to be recorded.

[0035]

[0036]

[0037] Then, the pixel value of each pixel in the output image is calculated by averaging or weighted averaging the pixel values ​​corresponding to the calculated coordinate values.

[0038] f(O x )=f(I x )+w1×f(I x ) (5)

[0039] f(O y )=f(I y )+w2×f(I y ) (6)

[0040] From this we can see that compared with pixel value calculation, the coordinate value calculation process is more complicated and often takes more calculation time.

[0041] That is to say, common resizing methods include both coordinate value and pixel value calculation processes in the inference process, and the coordinate value calculation, whether it is the nearest neighbor, bilinear, or trilinear algorithm, involves a large amount of repeated calculations, which will increase unnecessary time consumption and reduce the processing efficiency of video super-resolution.

[0042] In order to solve the above problems, in an embodiment of the present application, the terminal device constructs a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple; when performing super-resolution processing on the original image, based on the mapping relationship between the super-resolution pixels and the reference pixels, the pixel value of the reference pixel corresponding to the current pixel in the original image is determined; the pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel. That is to say, in an embodiment of the present application, the terminal device can perform coordinate value calculation according to the super-resolution multiple in advance before starting to perform super-resolution processing on the original image, and complete the construction of the mapping relationship between the super-resolution pixels and the reference pixels, so that in the process of super-resolution processing, the mapping relationship between the super-resolution pixels and the reference pixels that has been constructed can be directly used to complete the calculation of the pixel value, so that the repeated and highly complex coordinate value calculation processing can be avoided. It can be seen that in the present application, the coordinate value calculation and pixel value calculation processes in the resize calculation are separated, and the mapping relationship between the super-resolution pixels and the reference pixels is determined in advance, thereby reducing the complexity and repeatability of the calculation in the super-resolution process, improving the processing efficiency of the video super-resolution, and thus improving the super-resolution performance of the terminal device.

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0044] An embodiment of the present application provides a super-resolution method, which can be applied to a terminal device. Figure 1 This is a schematic diagram of the implementation process of the super-resolution method proposed in the embodiment of the present application, such as Figure 1 As shown, the method for performing super-resolution on a terminal device may include the following steps:

[0045] Step 101: construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple.

[0046] In an embodiment of the present application, the terminal device may first construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple.

[0047] It should be noted that in the embodiments of the present application, the super-resolution multiple may be a super-resolution magnification multiple. Image super-resolution may be referred to as super-resolution, i.e., a method of resolution, and the super-resolution magnification multiple is used to describe the size relationship between the image before and after super-resolution.

[0048] It can be understood that, in the embodiment of the present application, assuming that the super-resolution multiple is k, it can be considered that the length and width of the image after super-resolution are respectively enlarged by k times compared with the image before super-resolution. Correspondingly, the area of ​​the image after super-resolution is k times the area of ​​the image before super-resolution. 2 times.

[0049] It should be noted that, in the embodiments of the present application, the commonly used super-resolution video method includes a resize calculation, wherein the resize calculation is used for image scaling processing, and mainly includes two steps of coordinate value calculation and pixel value calculation.

[0050] Accordingly, in the embodiments of the present application, for the same super-resolution algorithm, i.e., the same super-resolution multiple, the sizes of the image before super-resolution (i.e., the original image) and the image after super-resolution are both fixed, and it can be considered that the sizes of the images before and after the resize calculation are fixed. For example, when the super-resolution multiple is 2, the size of the original image before super-resolution is 512x512, and the size of the image after super-resolution is 1024x1024.

[0051] It is understandable that in the embodiments of the present application, precisely because the size of the image before and after the resizing calculation is fixed in the same super-resolution algorithm, it is possible to choose to execute the coordinate value calculation part with high complexity and repeatability in the resizing calculation in advance.

[0052] Further, in an embodiment of the present application, after determining the super-resolution multiple, a mapping relationship between super-resolution pixels and reference pixels can be constructed according to the super-resolution multiple. The super-resolution pixels can be pixels in the image after super-resolution, and the reference pixels can be pixels in the original image before super-resolution. Accordingly, the mapping relationship between the super-resolution pixels and the reference pixels can be used to determine the position coordinate relationship between the pixels in the image after super-resolution and the pixels in the original image.

[0053] That is to say, in the embodiments of the present application, the coordinate value calculation part in the resize calculation can be completed in advance, so as to construct a mapping relationship between super-resolution pixels and reference pixels, and then the coordinate value calculation process can be no longer repeated during the super-resolution processing.

[0054] Exemplarily, in some embodiments, during the initialization process of video super-resolution, processes such as constructing a mapping relationship between super-resolution pixels and reference pixels, allocating memory, and creating operators can be performed synchronously, thereby reducing the time consumption of the actual reasoning calculation process of video super-resolution.

[0055] Furthermore, in an embodiment of the present application, after the mapping relationship between the super-resolution pixels and the reference pixels is constructed according to the super-resolution multiple, the mapping relationship between the super-resolution pixels and the reference pixels may also be stored.

[0056] Exemplarily, in some embodiments, the mapping relationship between the super-resolution pixels and the reference pixels may be stored in an indirect buffer.

[0057] Furthermore, in an embodiment of the present application, the terminal device may first determine a coordinate value calculation type, wherein the coordinate value calculation type may include any one of the following calculation types: nearest neighbor interpolation calculation, bilinear interpolation calculation, and trilinear interpolation calculation.

[0058] Correspondingly, in an embodiment of the present application, when constructing a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple, at least one reference position corresponding to each pixel position in the super-resolution image can be determined according to the super-resolution multiple and the coordinate value calculation type; and then a mapping relationship between super-resolution pixels and reference pixels can be constructed according to at least one pixel point index corresponding to at least one reference position in the original image.

[0059] It can be understood that in the embodiments of the present application, for different coordinate value calculation types, the corresponding reference pixels determined may be different. For example, if the coordinate value calculation type used is the nearest neighbor interpolation calculation, then it is necessary to rely on 1 reference pixel. If the coordinate value calculation type used is the bilinear interpolation calculation, then it is necessary to rely on 4 reference pixels. If the coordinate value calculation type used is the trilinear interpolation calculation, then it is necessary to rely on 8 reference pixels.

[0060] It can be seen that in the embodiments of the present application, the super-resolution multiple and the coordinate value calculation type will affect the mapping relationship between the super-resolution pixel and the reference pixel. Therefore, the terminal device can first perform coordinate value calculation processing according to the super-resolution multiple and the coordinate value calculation type, and for each pixel position in the super-resolution image, determine at least one corresponding reference position, thereby completing the construction of the mapping relationship between the super-resolution pixel and the reference pixel.

[0061] Exemplarily, in some embodiments, assuming that the size of the original image is W i ×H i , the size of the super-resolution image is W o ×H o The image, then, the corresponding super-resolution multiple d is:

[0062] d=W o / W i (7)

[0063] Or:

[0064] d=H o / H i (8)

[0065] Exemplarily, in some embodiments, it is assumed that the position of the reference pixel in the original image before super-resolution is expressed as (I x , I y ), which is the input coordinate (I x , I y ), if the coordinate value calculation type is the nearest neighbor interpolation calculation, then the position coordinates of the super-resolution pixel in the corresponding super-resolution image are expressed as (0 x , O y ), that is, the output coordinates (O x , O y ) pixels, the calculation formula between the two is:

[0066] O x =I x ×d (9)

[0067] O y =I y ×d (10)

[0068] Among them, for the coordinate value calculation type of the nearest neighbor interpolation calculation, the pixel position (0 x , O y ) corresponds to a reference position (I x , I y ), that is, when the nearest neighbor interpolation is used for coordinate calculation, a pixel position in the super-resolution image corresponds to a reference position in the original image.

[0069] Accordingly, in the embodiment of the present application, for the coordinate value calculation type of the bilinear interpolation calculation, the pixel position (0 x , O y ) corresponds to the four reference positions, namely (I x , I y ), (I x , I y +1), (I x +1,I y ), (I x +1,I y +1), that is, when bilinear interpolation is used to calculate coordinate values, one pixel position in the super-resolution image corresponds to four reference positions in the original image.

[0070] Correspondingly, in an embodiment of the present application, when trilinear interpolation is used to calculate coordinate values, one pixel position in the super-resolution image corresponds to eight reference positions in the original image.

[0071] Furthermore, after determining at least one reference position corresponding to each pixel position in the super-resolution image according to the super-resolution multiple and the coordinate value calculation type, the mapping relationship between the super-resolution pixel and the reference pixel can be further constructed according to at least one pixel point index corresponding to at least one reference position in the original image.

[0072] That is to say, in an embodiment of the present application, a different pixel point index can be assigned to each reference position in the original image, so that when constructing a mapping relationship between super-resolution pixels and reference pixels, a correspondence between the super-resolution pixels and the pixel point index of at least one corresponding reference pixel can be established.

[0073] Exemplarily, in some embodiments, Table 1 is a mapping relationship between super-resolution pixels and reference pixels corresponding to nearest neighbor interpolation. It is assumed that the mapping relationship between super-resolution pixels and reference pixels is expressed as a corresponding relationship between the pixel position after super-resolution and the pixel point index of the reference pixel. For example, the pixel position (1, 0) corresponds to the reference pixel with an index of 1.

[0074] Table 1

[0075] Pixel location Pixel index (1，0) 1 (1，1) 2 (2，0) 3 (2，1) 4 …… …… (2，2) 8

[0076] Exemplarily, in some embodiments, Table 2 is a mapping relationship between super-resolution pixels and reference pixels corresponding to bilinear interpolation. It is assumed that the mapping relationship between super-resolution pixels and reference pixels is expressed as a corresponding relationship between the pixel position after super-resolution and the pixel point index of the reference pixel. For example, the pixel position (1, 1) corresponds to reference pixels with indices 2, 3, 4, and 5.

[0077] Table 2

[0078] Pixel location Pixel index (1，0) 1，2，3，4 (1，1) 2，3，4，5 …… …… (2，2) 5，6，7，8

[0079] It should be noted that in the embodiments of the present application, when constructing the mapping relationship between super-resolution pixels and reference pixels, the present application does not limit the form of the mapping relationship, which can be either a correspondence between the pixel point index of the super-resolution pixel and the corresponding reference pixel, or a correspondence between the identification signal of the super-resolution pixel and the corresponding reference pixel.

[0080] It can be understood that in an embodiment of the present application, when storing the mapping relationship between super-resolution pixels and reference pixels in the indirect cache, the reference pixels in the original image can be stored in the form of an array, wherein the pixel point index corresponding to the reference pixel is used to determine the position of the reference pixel in the array.

[0081] Exemplarily, in some embodiments, Figure 2 It is a schematic diagram of the mapping relationship between super-resolution pixels and reference pixels, such as Figure 2 As shown in the figure, if bilinear interpolation is used to calculate the coordinate value, then for the pixel position of a super-resolution pixel after super-resolution processing, it corresponds to the pixel positions of 4 reference pixels in the original image. For example, for the pixel point at the pixel position (2, 1) in the super-resolution image, it depends on the pixel points of the 4 coordinates (1, 0), (2, 0), (1, 1), and (2, 1) in the original image. Then, the mapping relationship between the super-resolution pixel and the reference pixel can be stored in the indirect cache.

[0082] Step 102: When performing super-resolution processing on the original image, based on the mapping relationship between the super-resolution pixels and the reference pixels, determine the pixel value of the reference pixel corresponding to the current pixel in the original image.

[0083] In an embodiment of the present application, after constructing a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple, when the original image is subsequently subjected to super-resolution processing, the pre-stored mapping relationship between super-resolution pixels and reference pixels can be obtained first, and then based on the mapping relationship between the super-resolution pixels and the reference pixels, the pixel value of the reference pixel corresponding to the current pixel in the original image can be determined.

[0084] Furthermore, in an embodiment of the present application, when determining the pixel value of a reference pixel corresponding to a current pixel in an original image based on a mapping relationship between super-resolution pixels and reference pixels, the index of the reference pixel corresponding to the current pixel can be first determined based on the mapping relationship between the super-resolution pixels and the reference pixels and the pixel position of the current pixel; and then the pixel value of the reference pixel can be determined in the original image based on the index of the reference pixel.

[0085] It can be understood that in an embodiment of the present application, when super-resolution processing is performed on the original image, the mapping relationship between the super-resolution pixels and the reference pixels pre-stored in the indirect cache can be directly obtained, so that based on the mapping relationship between the super-resolution pixels and the reference pixels, the index corresponding to the reference pixel on which the current pixel depends can be determined according to the pixel position of the current pixel after super-resolution.

[0086] It should be noted that, in the embodiments of the present application, the index of the reference pixel can be used to indicate the address information of the reference pixel; or, the index of the reference pixel can also be used to indicate the offset information between the address of the reference pixel and the initial address.

[0087] It can be understood that in the embodiments of the present application, when the mapping relationship between the super-resolution pixels and the reference pixels is stored in the indirect cache, it can be considered that what is stored is the pointer of each reference pixel, that is, the address information of each reference pixel is stored; it can also be considered that what is stored is the offset of each reference pixel and the relative first address (initial address), that is, the offset information of the address of each reference pixel and the initial address is stored.

[0088] That is to say, in the embodiment of the present application, in the indirect cache, in theory, the pointers of each pixel of the input image (original image) are stored, but considering the limited size of the input image, the offset of the pixel and its relative first address can also be stored in integer type to replace the pointer storage method. Accordingly, in the process of pixel value calculation, the pixel value of the input image can be read by the first address + offset method to complete the calculation.

[0089] Exemplarily, in some embodiments, Figure 3 A schematic diagram of replacing pointers with integers, such as Figure 3 As shown in the figure, in the indirect cache, theoretically, pointers to each point of the input image are stored. On a 64-bit system, the pointer is stored in 64 bits (8 bytes). Considering the limited size of the input image, a 32-bit integer type can be used to represent its offset relative to the first address to replace the pointer storage method. Among them, storing 32-bit (4-byte) integer values ​​can save half of the buffer (indirect cache) space.

[0090] Exemplarily, in some embodiments, Table 3 is a mapping relationship between super-resolution pixels and reference pixels corresponding to nearest neighbor interpolation. Assuming that the mapping relationship between super-resolution pixels and reference pixels is expressed as a corresponding relationship between pixel positions after super-resolution and pixel point indices of reference pixels, then, when super-resolution processing is performed on the original image, the index of the corresponding reference pixel is determined according to the pixel position of the current pixel in the super-resolution image. For example, for the current pixel with a pixel position of (1, 1), the index of the corresponding reference pixel can be determined to be b according to the mapping relationship between the super-resolution pixels and the reference pixels, and then the pixel value of the corresponding reference pixel can be determined based on the index.

[0091] Table 3

[0092] Pixel location Pixel index (1，0) a (1，1) b (2，0) c (2，1) d …… …… (2，2) h

[0093] Exemplarily, in some embodiments, Table 4 is a mapping relationship between super-resolution pixels and reference pixels corresponding to bilinear interpolation. Assuming that the mapping relationship between super-resolution pixels and reference pixels is expressed as a corresponding relationship between pixel positions after super-resolution and pixel point indices of reference pixels, when super-resolution processing is performed on the original image, the index of the corresponding reference pixel is determined according to the pixel position of the current pixel in the super-resolution image. For example, for the current pixel with a pixel position of (1, 0), the index of the corresponding reference pixel can be determined as a, b, c, d according to the mapping relationship between the super-resolution pixels and the reference pixels, and then the pixel value of the corresponding reference pixel can be determined based on the index.

[0094] Table 4

[0095] Pixel location Pixel index (1，0) a, b, c, d (1，1) a, b, c, e …… …… (2，2) b, c, e, h

[0096] It can be understood that in the embodiments of the present application, since the pixel point index corresponding to the reference pixel can be used to determine the position of the reference pixel in the array, after determining the index of the reference pixel corresponding to the current pixel based on the mapping relationship between the current pixel and the super-resolution pixel and the reference pixel, the pixel value of the reference pixel can be further obtained from the original image based on the index of the reference pixel.

[0097] Furthermore, in an embodiment of the present application, when determining the pixel value of a reference pixel corresponding to a current pixel in an original image based on a mapping relationship between super-resolution pixels and reference pixels, the vector index corresponding to the current pixel can be first determined based on the mapping relationship between super-resolution pixels and reference pixels and the pixel position of the current pixel; then, based on the vector index, the vectorized features of the reference pixel can be determined; finally, the pixel value of the reference pixel can be determined based on the vectorized features.

[0098] It should be noted that in an embodiment of the present application, in the process of acquiring the pixel value of the reference pixel, it is also possible to first acquire the vectorized features of the reference pixel based on the mapping relationship between the super-resolution pixel and the reference pixel, and then use the vectorized features of the reference pixel to further determine the pixel value of the reference pixel.

[0099] It can be understood that in the embodiments of the present application, the pre-constructed mapping relationship between super-resolution pixels and reference pixels can also be expressed as a corresponding relationship between the pixel position after super-resolution and the vector index of the vectorized feature corresponding to the reference pixel.

[0100] Exemplarily, in some embodiments, Table 5 is a mapping relationship between super-resolution pixels and reference pixels corresponding to bilinear interpolation. Assuming that the mapping relationship between super-resolution pixels and reference pixels is expressed as a corresponding relationship between pixel positions after super-resolution and vector indices of reference pixels, when super-resolution processing is performed on the original image, the vector index corresponding to the vector feature of the corresponding reference pixel is determined according to the pixel position of the current pixel in the super-resolution image. For example, for the current pixel with a pixel position of (1, 0), the vector index of the corresponding reference pixel can be determined to be 1 according to the mapping relationship between the super-resolution pixels and the reference pixels, and then the vectorized feature of the corresponding reference pixel can be determined based on the vector index.

[0101] Table 5

[0102] Pixel location Vector Indexing (1，0) 1 (1，1) 2 …… …… (2，2) 8

[0103] It is understood that, in the embodiment of the present application, the reference pixel may be vectorized to obtain the corresponding vectorized feature. The pixel value of the reference pixel may be vectorized using a vector processor to generate the corresponding vectorized feature. The length of the specific vector is determined according to the characteristics of the processor and the type of the calculated value.

[0104] Exemplarily, in some embodiments, Figure 4 This is a schematic diagram of vectorized calculation, such as Figure 4 As shown, in the process of using the pixel values ​​of the reference pixels in the four original images to calculate the pixel value of the current pixel in the super-resolution image, the vectorization characteristics of the vector processor can be used to perform vectorization processing between the four reference pixels and in the channel dimension, specifically including data extraction and vector calculation. The length of the vector is determined according to the characteristics of the processor and the type of numerical value calculated.

[0105] Step 103: Determine the pixel value of the current pixel according to the pixel value of the reference pixel, and determine the super-resolution image corresponding to the original image based on the pixel value of the current pixel.

[0106] In an embodiment of the present application, when super-resolution processing is performed on the original image, after determining the pixel value of the reference pixel corresponding to the current pixel in the original image based on the mapping relationship between the super-resolution pixels and the reference pixels, the pixel value of the current pixel can be further determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image can be determined based on the pixel value of the current pixel.

[0107] It can be understood that in the implementation of the present application, after obtaining the pixel value of the reference pixel corresponding to the current pixel based on the mapping relationship between the super-resolution pixel and the reference pixel, the pixel value of the reference pixel can be used to calculate the pixel value of the current pixel, so as to determine the pixel value of the current pixel.

[0108] It can be understood that in the embodiments of the present application, pixel value calculation may include but is not limited to average calculation or weighted average calculation, and the present application does not impose any specific limitation.

[0109] Furthermore, in an embodiment of the present application, when determining the pixel value of the current pixel based on the pixel value of the reference pixel, the pixel weight can be first determined based on the pixel position and super-resolution multiple of the current pixel; and then the pixel value of the current pixel can be determined based on the pixel value and pixel weight of the reference pixel.

[0110] It is understood that in the embodiment of the present application, if the pixel value of the current pixel is determined by weighted average calculation based on the pixel value of the reference pixel, it is necessary to first determine the pixel weight of the weighted average calculation. The pixel weight can be determined by the pixel position of the current pixel and the super-resolution multiple of the super-resolution processing.

[0111] Exemplarily, in some embodiments, assuming that the super-resolution multiple is d, the pixel position of the current pixel in the super-resolution image is (0 x , O y ), then the pixel weights, w1, w2, can be determined according to the pixel position of the current pixel and the super-resolution multiple for super-resolution processing.

[0112]

[0113]

[0114] For example, in some embodiments, if the coordinate value calculation type is the nearest neighbor interpolation calculation, the current pixel (0 x , O y ) is represented by the pixel value f( x ) and f(O y ), the corresponding reference pixel (I x , I y ) is represented by the pixel value f(I x ) and f(I y ), then the pixel value of the current pixel can be calculated based on the pixel value and pixel weight of the reference pixel, as shown in the above formulas (5) and (6).

[0115] Furthermore, in the present application, after the pixel value of the current pixel is determined according to the pixel value of the reference pixel, the super-resolution image corresponding to the original image can be determined based on the pixel value of the current pixel.

[0116] It is understandable that in the embodiments of the present application, each pixel position in the super-resolution image may be traversed to determine the pixel value corresponding to each pixel position, thereby obtaining the super-resolution image.

[0117] That is to say, in an embodiment of the present application, when calculating pixel values, the pre-stored mapping relationship between super-resolution pixels and reference pixels can be repeatedly read to determine the pixel value corresponding to each pixel position in turn, thereby completing the determination of the entire super-resolution image.

[0118] For example, in the embodiments of the present application, Figure 5 Schematic diagram of the pixel value calculation process Figure 1 ,like Figure 5 As shown, the indirect cache has stored the mapping relationship between the super-resolution pixels and the reference pixels pre-built at the time of initialization, so that in the process of calculating the pixel value, the index of the reference pixel corresponding to the current pixel can be determined from the mapping relationship between the super-resolution pixels and the reference pixels stored in the indirect cache, and then the pixel value of the corresponding reference pixel can be determined in the original image according to the index, so that the pixel value of the current pixel can be calculated according to the pixel value of the reference pixel. Repeating the above process continuously, that is, continuously using the mapping relationship between the super-resolution pixels and the reference pixels pre-built and stored in the indirect cache, the pixel value corresponding to each pixel position is determined in turn, and finally a super-resolution image can be obtained.

[0119] In summary, through the super-resolution method proposed in the above steps 101 to 103, before performing super-resolution processing on the original image, the coordinate value calculation processing can be completed in advance according to the super-resolution multiple, and the mapping relationship between the super-resolution pixels and the reference pixels can be constructed. Therefore, in the subsequent pixel value calculation process of the super-resolution processing, it is only necessary to read the mapping relationship between the super-resolution pixels and the reference pixels from the indirect conversion to determine the corresponding reference pixels, and no longer perform coordinate value calculation processing with high repetitiveness and complexity.

[0120] It can be understood that the super-resolution method proposed in the embodiment of the present application, on the one hand, uses an indirect buffer method to separate the coordinate value calculation and pixel value calculation processes in the resize calculation, and completes the former in the initialization process before the square resolution processing, thereby improving the computing performance of the running process.

[0121] It can be understood that the super-resolution method proposed in the embodiment of the present application, on the other hand, makes full use of the parallel computing characteristics of vector processors, such as graphics processing units (NPUs), and multi-threaded processors to vectorize and parallelize multi-channel data, thereby avoiding waste of computing resources and further improving computing performance.

[0122] It can be understood that the super-resolution method proposed in the embodiment of the present application, on the other hand, uses address values ​​instead of pointers, which can save 50% of the memory space occupied by the indirect cache in a 64-bit system without affecting the calculation results of the algorithm.

[0123] It should be noted that the super-resolution method proposed in the embodiment of the present application is not limited to super-resolution neural networks, and any neural network that includes resizing calculations and has relatively high requirements for performance and power consumption can use this solution.

[0124] It should be noted that the super-resolution method proposed in the embodiment of the present application is not limited to being deployed on mobile phones. Any smart device with a CPU, GPU or NPU chip can be deployed using this solution.

[0125] It should be noted that the super-resolution method proposed in the embodiment of the present application is not limited to use in AI algorithms. Traditional CV algorithms can also use this solution, and it is only necessary to replace the AI ​​framework in this solution.

[0126] It can be seen that the super-resolution method proposed in the embodiment of the present application can reduce the amount of calculation in the reasoning process, thereby effectively improving the performance of the resizing calculation process in the super-resolution algorithm, and give full play to the characteristics of vectorized computing and parallel computing of computing chips, so that the super-resolution algorithm has a higher bandwidth, thereby improving performance such as frame rate, or supporting higher resolution calculations.

[0127] The embodiment of the present application provides a super-resolution method, wherein the terminal device constructs a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple; when super-resolution processing is performed on the original image, the pixel value of the reference pixel corresponding to the current pixel is determined in the original image based on the mapping relationship between the super-resolution pixels and the reference pixels; the pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel. That is to say, in the embodiment of the present application, the terminal device can perform coordinate value calculation according to the super-resolution multiple in advance before starting to perform super-resolution processing on the original image, and complete the construction of the mapping relationship between the super-resolution pixels and the reference pixels, so that in the process of super-resolution processing, the mapping relationship between the super-resolution pixels and the reference pixels that has been constructed can be directly used to complete the calculation of the pixel value, so that the repeated and highly complex coordinate value calculation processing can be avoided. It can be seen that in the present application, the coordinate value calculation and pixel value calculation processes in the resize calculation are separated, and the mapping relationship between the super-resolution pixels and the reference pixels is determined in advance, thereby reducing the complexity and repeatability of the calculation in the super-resolution process, improving the processing efficiency of the video super-resolution, and thus improving the super-resolution performance of the terminal device.

[0128] Based on the above embodiments, in another embodiment of the present application, the resize calculation involved in the video super-resolution algorithm can be divided into two steps: coordinate value calculation and pixel value calculation. Among them, compared with the pixel value calculation, the coordinate value calculation process is more complicated, the calculation time occupied will often be more, and the calculation process for different frames is the same, which belongs to repeated calculation. Considering that in the same super-resolution algorithm, the image size before and after resizing is fixed, that is, the coordinate value calculation process, the result is the same in each reasoning calculation process. Therefore, an embodiment of the present application proposes a super-resolution method, which puts the pixel value calculation process into the initialization process, and performs it together with memory allocation, operator creation and other processes, thereby reducing the time-consuming actual reasoning calculation process of video super-resolution.

[0129] Exemplarily, in some embodiments, Figure 6 This is a schematic diagram of the coordinate value calculation process, such as Figure 6 As shown in the figure, taking the bilinear interpolation method as an example, in the process of resizing a 3x3 image to a 4x4 image, that is, the super-resolution multiple is 4 / 3, each output pixel (super-resolution pixel) in the output image (super-resolution image) depends on the values ​​of 4 input pixels (reference pixels) in the input image (original image), and the addresses (i.e., indexes) of the corresponding 4 pixels can be stored in an indirect buffer. Because the space allocated for the input image is fixed, the addresses stored in the buffer are also fixed during the super-resolution operation of the video stream.

[0130] For example, in some embodiments, during initialization, the positions of the four input points corresponding to each output point may be calculated and stored in a buffer.

[0131] Exemplarily, in some embodiments, Figure 7 Schematic diagram of the pixel value calculation process Figure 2 ,like Figure 7 It is shown that the pixel at the coordinate (2, 1) of the output image depends on the four points (1, 0), (2, 0), (1, 1), and (2, 1) in the input image through the above formula. Therefore, when running the inference calculation, the four pixel values ​​are read directly from the buffer storage location, and the calculation of the corresponding pixel values ​​(such as calculating the average or weighted average) is completed. This process is repeated to finally obtain the output image.

[0132] For example, in some embodiments, there are 4 input points for the bilinear interpolation method, and if it is the nearest interpolation method, one output point depends on only one input point, while the cubic linear interpolation method depends on 8 points in three-dimensional space.

[0133] For example, in some embodiments, Figure 4 As shown in the figure, in the process of calculating 1 output pixel value from 4 input pixel values, the vectorization characteristics of the vector processor can be used to perform vectorization processing between the 4 input points and in the channel dimension. The specific length of the vector is determined according to the characteristics of the processor and the type of value calculated.

[0134] For example, in some embodiments, Figure 3 As shown in the figure, in the indirect cache, theoretically, pointers to each point of the input image are stored. However, considering the limited size of the input image, a 32-bit integer type can be used to represent its offset relative to the first address to replace the pointer storage method. In the pixel value calculation process, the pixel value of the input image is read by the first address + offset method to complete the calculation. On a 64-bit system, pointers are stored using 64 bits (8 bytes), so storing 32-bit (4-byte) integer values ​​can save half of the buffer space.

[0135] To summarize, the super-resolution method proposed in the embodiment of the present application can complete the coordinate value calculation processing according to the super-resolution multiple in advance before performing the super-resolution processing on the original image, and construct the mapping relationship between the super-resolution pixels and the reference pixels. Therefore, in the pixel value calculation process of the subsequent super-resolution processing, it is only necessary to read the mapping relationship between the super-resolution pixels and the reference pixels from the indirect conversion to determine the corresponding reference pixels, and no longer perform the coordinate value calculation processing with high repetitiveness and complexity.

[0136] It can be understood that the super-resolution method proposed in the embodiment of the present application, on the one hand, uses an indirect buffer method to separate the coordinate value calculation and pixel value calculation processes in the resize calculation, and completes the former in the initialization process before the square resolution processing, thereby improving the computing performance of the running process.

[0137] It can be understood that the super-resolution method proposed in the embodiment of the present application, on the other hand, makes full use of the parallel computing characteristics of vector processors, such as graphics processing units (NPUs), and multi-threaded processors to vectorize and parallelize multi-channel data, thereby avoiding waste of computing resources and further improving computing performance.

[0138] It can be understood that the super-resolution method proposed in the embodiment of the present application, on the other hand, uses address values ​​instead of pointers, which can save 50% of the memory space occupied by the indirect cache in a 64-bit system without affecting the calculation results of the algorithm.

[0139] It should be noted that the super-resolution method proposed in the embodiment of the present application is not limited to super-resolution neural networks, and any neural network that includes resizing calculations and has relatively high requirements for performance and power consumption can use this solution.

[0140] It should be noted that the super-resolution method proposed in the embodiment of the present application is not limited to being deployed on mobile phones. Any smart device with a CPU, GPU or NPU chip can be deployed using this solution.

[0141] It should be noted that the super-resolution method proposed in the embodiment of the present application is not limited to use in AI algorithms. Traditional CV algorithms can also use this solution, and it is only necessary to replace the AI ​​framework in this solution.

[0142] It can be seen that the super-resolution method proposed in the embodiment of the present application can reduce the amount of calculation in the reasoning process, thereby effectively improving the performance of the resizing calculation process in the super-resolution algorithm, and give full play to the characteristics of vectorized computing and parallel computing of computing chips, so that the super-resolution algorithm has a higher bandwidth, thereby improving performance such as frame rate, or supporting higher resolution calculations.

[0143] The embodiment of the present application provides a super-resolution method, wherein the terminal device constructs a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple; when super-resolution processing is performed on the original image, the pixel value of the reference pixel corresponding to the current pixel is determined in the original image based on the mapping relationship between the super-resolution pixels and the reference pixels; the pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel. That is to say, in the embodiment of the present application, the terminal device can perform coordinate value calculation according to the super-resolution multiple in advance before starting to perform super-resolution processing on the original image, and complete the construction of the mapping relationship between the super-resolution pixels and the reference pixels, so that in the process of super-resolution processing, the mapping relationship between the super-resolution pixels and the reference pixels that has been constructed can be directly used to complete the calculation of the pixel value, so that the repeated and highly complex coordinate value calculation processing can be avoided. It can be seen that in the present application, the coordinate value calculation and pixel value calculation processes in the resize calculation are separated, and the mapping relationship between the super-resolution pixels and the reference pixels is determined in advance, thereby reducing the complexity and repeatability of the calculation in the super-resolution process, improving the processing efficiency of the video super-resolution, and thus improving the super-resolution performance of the terminal device.

[0144] Based on the above embodiment, in another embodiment of the present application, Figure 8 Schematic diagram of the structure of the terminal device proposed in the embodiment of the present application Figure 1 ,like Figure 8 As shown, the terminal device 10 proposed in the embodiment of the present application may include a construction unit 111, a determination unit 112,

[0145] The construction unit 111 is used to construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple;

[0146] The determination unit 112, when performing super-resolution processing on the original image, determines the pixel value of the reference pixel corresponding to the current pixel in the original image based on the mapping relationship between the super-resolution pixel and the reference pixel; determines the pixel value of the current pixel according to the pixel value of the reference pixel, and determines the super-resolution image corresponding to the original image based on the pixel value of the current pixel.

[0147] In the embodiments of the present application, further, Fig. 9 Schematic diagram of the structure of the terminal device proposed in the embodiment of the present application Figure 2 ,like Fig. 9 As shown, the terminal device 10 proposed in the embodiment of the present application may also include a processor 121 and a memory 122 storing executable instructions of the processor 121. Further, the software testing device 10 may also include a communication interface 123 and a bus 124 for connecting the processor 121, the memory 122 and the communication interface 123.

[0148] In the embodiment of the present application, the above-mentioned processor 121 can be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other, and the embodiment of the present application is not specifically limited. The software testing device 10 can also include a memory 122, which can be connected to the processor 121, wherein the memory 122 is used to store executable program code, the program code includes computer operation instructions, and the memory 122 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.

[0149] In the embodiment of the present application, the bus 124 is used to connect the communication interface 123, the processor 121 and the memory 122, and the mutual communication between these devices.

[0150] In the embodiment of the present application, the memory 122 is used to store instructions and data.

[0151] Furthermore, in an embodiment of the present application, the above-mentioned processor 121 is used to construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple; when super-resolution processing is performed on the original image, based on the mapping relationship between the super-resolution pixels and the reference pixels, the pixel value of the reference pixel corresponding to the current pixel is determined in the original image; the pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel.

[0152] In practical applications, the memory 122 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or a combination of the above types of memories, and provide instructions and data to the processor 121.

[0153] In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.

[0154] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., which can store program code.

[0155] The embodiment of the present application provides a terminal device, which constructs a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple; when performing super-resolution processing on the original image, based on the mapping relationship between the super-resolution pixels and the reference pixels, the pixel value of the reference pixel corresponding to the current pixel in the original image is determined; the pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel. That is to say, in the embodiment of the present application, the terminal device can perform coordinate value calculation according to the super-resolution multiple in advance before starting to perform super-resolution processing on the original image, and complete the construction of the mapping relationship between the super-resolution pixels and the reference pixels, so that in the process of super-resolution processing, the mapping relationship between the super-resolution pixels and the reference pixels that has been constructed can be directly used to complete the calculation of the pixel value, so that the repeated and highly complex coordinate value calculation processing can be avoided. It can be seen that in the present application, the coordinate value calculation and pixel value calculation processes in the resize calculation are separated, and the mapping relationship between the super-resolution pixels and the reference pixels is determined in advance, thereby reducing the complexity and repeatability of the calculation in the super-resolution process, improving the processing efficiency of the video super-resolution, and thus improving the super-resolution performance of the terminal device.

[0156] An embodiment of the present application provides a computer-readable storage medium having a program stored thereon, which implements the super-resolution method as described above when executed by a processor.

[0157] Specifically, the program instructions corresponding to a super-resolution method in this embodiment may be stored on a storage medium such as an optical disk, a hard disk, or a USB flash drive. When the program instructions corresponding to a super-resolution method in the storage medium are read or executed by an electronic device, the following steps are included:

[0158] Construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple;

[0159] When super-resolution processing is performed on the original image, based on the mapping relationship between the super-resolution pixels and the reference pixels, a pixel value of a reference pixel corresponding to a current pixel in the original image is determined;

[0160] The pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel.

[0161] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0162] The present application is described with reference to implementation flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0163] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the implementation flow diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing the steps in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0165] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A super-resolution method, characterized in that: The method comprises: Construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple; When super-resolution processing is performed on the original image, based on the mapping relationship between the super-resolution pixels and the reference pixels, a pixel value of a reference pixel corresponding to a current pixel in the original image is determined; The pixel value of the current pixel is determined according to the pixel value of the reference pixel, and the super-resolution image corresponding to the original image is determined based on the pixel value of the current pixel.

2. The method according to claim 1, characterized in that The method further comprises: Determine a coordinate value calculation type; wherein the coordinate value calculation type includes any one of the following calculation types: nearest neighbor interpolation calculation, bilinear interpolation calculation, and trilinear interpolation calculation.

3. The method according to claim 2, characterized in that The step of constructing a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple includes: Determining at least one reference position corresponding to each pixel position in the super-resolution image according to the super-resolution multiple and the coordinate value calculation type; A mapping relationship between the super-resolution pixel and the reference pixel is constructed according to at least one pixel point index corresponding to the at least one reference position in the original image.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: The mapping relationship between the super-resolution pixels and the reference pixels is stored in an indirect cache.

5. The method according to claim 4, characterized in that The determining, based on the mapping relationship between the super-resolution pixel and the reference pixel, a pixel value of a reference pixel corresponding to a current pixel in the original image includes: Determine the index of the reference pixel corresponding to the current pixel according to the mapping relationship between the super-resolution pixel and the reference pixel and the pixel position of the current pixel; According to the index of the reference pixel, a pixel value of the reference pixel is determined in the original image.

6. The method according to claim 1, characterized in that The determining, based on the mapping relationship between the super-resolution pixel and the reference pixel, a pixel value of a reference pixel corresponding to a current pixel in the original image includes: Determine a vector index corresponding to the current pixel according to a mapping relationship between the super-resolution pixel and the reference pixel and a pixel position of the current pixel; Determining a vectorized feature of the reference pixel according to the vector index; A pixel value of the reference pixel is determined according to the vectorized feature.

7. The method according to claim 5 or 6, characterized in that: The determining the pixel value of the current pixel according to the pixel value of the reference pixel comprises: Determining a pixel weight according to a pixel position of the current pixel and the super-resolution multiple; The pixel value of the current pixel is determined according to the pixel value of the reference pixel and the pixel weight.

8. The method according to claim 5, characterized in that The index of the reference pixel is used to indicate the address information of the reference pixel; or, The index of the reference pixel is used to indicate the offset information between the address of the reference pixel and the initial address.

9. The method according to any one of claims 1 to 8, characterized in that: The step of determining the super-resolution image corresponding to the original image based on the pixel value of the current pixel includes: Each pixel position in the super-resolution image is traversed, and a pixel value corresponding to each pixel position is determined to obtain the super-resolution image.

10. A terminal device, characterized in that: The terminal device comprises: a construction unit, a determination unit, The construction unit is used to construct a mapping relationship between super-resolution pixels and reference pixels according to the super-resolution multiple; The determination unit, when performing super-resolution processing on the original image, determines the pixel value of the reference pixel corresponding to the current pixel in the original image based on the mapping relationship between the super-resolution pixel and the reference pixel; determines the pixel value of the current pixel according to the pixel value of the reference pixel, and determines the super-resolution image corresponding to the original image based on the pixel value of the current pixel.

11. A terminal device, characterized in that: The terminal device comprises: a processor and a memory; wherein, The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the method according to any one of claims 1 to 9 when running the computer program.

12. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images