Training method of super-resolution model, super-resolution method and system

Abstract

A method for training a super-resolution model, and a super-resolution method and system implemented by the artificial intelligence super-resolution model obtained through this training method, wherein the training method provides an input image and sets a non-integer magnification and an image quality threshold, then obtains the pixel values ​​of the input image and acquires image features, and generates multiple channel images through a super-resolution model based on the image features and the non-integer magnification of the input image, then obtains a mask corresponding to each channel image based on the phase information obtained by comparing the position of the output pixels with the magnification, then applies the corresponding multiple masks to the multiple channel images and reconstructs an output image, and evaluates the model parameters of the output image by comparing with the image quality threshold, so as to train the artificial intelligence super-resolution model.

Description

Technical Field

[0001] The specification discloses a technique for obtaining large-size images through super-resolution algorithms, and in particular a training method, super-resolution method and system for training a non-integer magnification super-resolution model through masking and neural network-like techniques. Background Technology

[0002] Super resolution is a computational technique that can improve image resolution. It is often combined with artificial intelligence (AI) techniques to use machine learning algorithms to enhance image resolution, known as Artificial Intelligence Super Resolution (AISR) algorithms. The technical concept involves using a large number of example images to train a super resolution machine learning model based on a convolutional neural network. The trained model matches low-resolution images with high-resolution images; for example, it can upscale a 10-megapixel image to a 40-megapixel image while preserving rich details. The operation of existing super resolution algorithms can be found in [reference needed]. Figure 1 The conceptual diagram shown has an input image 101 that is a low-resolution image. After being processed by the super-resolution machine learning model 103, the output image 105 is the calculated high-resolution image.

[0003] Currently, existing AISR (Artificial Intelligence Super Resolution) algorithm technology and hardware design mainly focus on integer magnification (1x, 2x, 3x, ...). Some other technologies have mentioned how to use algorithms to handle arbitrary magnification.

[0004] Regarding AI super-resolution algorithms for integer multiples, some papers have proposed enhanced deep residual networks for single image super resolution, and the proposed model architecture can achieve single-scale super-resolution (EDSR) that can improve computational efficiency.

[0005] Regarding methods for performing super-resolution algorithms at arbitrary magnification, a proposed method is a magnification-arbitrary network for super resolution. This method implements super-resolution algorithms at arbitrary scales using only a single model. By inputting scale factors and using dynamically predicted weights of upscale filters, low-resolution images can be transformed into high-resolution images of arbitrary sizes.

[0006] In existing neural network-based hardware designs, input and output information need to be processed using pipelines. In practical applications, when the output / input ratio is not an integer, there are limitations that prevent the direct use of AI super-resolution technology to improve image quality. Instead, a scaler needs to be added in the latter half of the calculation process to interpolate and compensate for the missing magnification.

[0007] For related algorithm diagrams, please refer to Figure 2 Based on the pipeline hardware design for implementing the aforementioned super-resolution algorithm, the super-resolution machine learning model 203 includes an image up-scaling converter in addition to the super-resolution model (SR model). Examples of integer magnification can be found in [reference needed]. Figure 3 The diagram shows the timing of the super-resolution algorithm running through hardware. When the input image 201 for which the super-resolution algorithm is applied is 10 (pixels) x 10 (rows) in size, and the scaling factor is set to 2x, it is converted into a 2x image through the image magnification function in the super-resolution machine learning model 203. 2 (4) Input small images, as shown in the figure, are single-line single-pixel image 211, single-line double-pixel image 212, double-line single-pixel image 213, and double-line double-pixel image 214. These are then reassembled into a large image, which in this example displays an output image 20 (pixels) x 20 (rows) 205. That is, refer to... Figure 3 The timing diagram shows that when an input image signal 301 with a 10-pixel line is input within the same time period, after passing through the super-resolution machine learning model 203, two output image signals 305 with 20 pixels each will be output.

[0008] Furthermore, for super-resolution algorithms that handle non-integer magnification, since traditional AI super-resolution algorithms cannot achieve non-integer magnification, a two-stage approach is adopted first, based on... Figure 4This diagram illustrates an example of an existing super-resolution algorithm that implements non-integer magnification. For instance, the input image 401 is 10 pixels x 10 rows in size, with a scale factor of 1.5x. First, the super-resolution machine learning model 403 executes the first stage of the super-resolution algorithm, outputting a first output image 405. Then, it is converted into a second output image 409 with non-integer magnification by an image magnification converter 407, such as an image of 15 pixels x 15 rows at a scale factor of 1.5x.

[0009] In actual operation, taking an input image of 10 pixels x 10 rows as an example, the input image 401 can be transformed into a first output image 405 of the same size (10 pixels x 10 rows) through the super-resolution machine learning model 403. Then, it is enlarged and transformed into a second output image 409 of 15 pixels x 15 rows. Alternatively, the super-resolution machine learning model 403 can first perform a 2x magnification to form a first output image 405 of 20 pixels x 20 rows, and then reduce the magnification to form a second output image 409 of 15 pixels x 15 rows.

[0010] However, existing non-integer magnification super-resolution algorithms still suffer from missing pixels due to poor magnification or reduction performance of the second-stage image upscaling converter 407. Summary of the Invention

[0011] To realize a non-integer magnification super-resolution algorithm and address the problem that existing non-integer magnification super-resolution algorithms still require traditional image magnification converters, this publication proposes a training method for a super-resolution model, a super-resolution method and system, and a super-resolution algorithm and system applying the artificial intelligence super-resolution model therein. This enables a non-integer magnification artificial intelligence super-resolution algorithm, in which the masking information required for image magnification conversion is obtained based on image information, and a super-resolution machine learning model is trained accordingly, resulting in larger, higher-resolution images with better performance.

[0012] According to an embodiment of a training method for a super-resolution model of artificial intelligence with non-integer magnification, an input image is provided, and a magnification and an image quality threshold are set. After obtaining the pixel values ​​of the input image, image features are acquired. Based on the image features of the input image and the set magnification, multiple channel images are obtained through a super-resolution model. Then, based on the magnification, phase information obtained by comparing the position of the output pixels is used to obtain the mask corresponding to each channel image. After applying the corresponding multiple masks to the multiple channel images, an output image is reconstructed. Then, the multiple masks of the obtained output image can be evaluated by comparing with the set image quality threshold, thereby training an artificial intelligence super-resolution model.

[0013] Preferably, during the training process, the model parameters of the non-integer magnification AI super-resolution model are updated through iterative procedures by repeating the above steps, so that the output image meets the image quality threshold.

[0014] Furthermore, different input images can be repeatedly input, and by using a large number of images and iterative procedures, the model parameters of the artificial intelligence super-resolution model can be converged.

[0015] Preferably, the above model parameters can be implemented as the weight values ​​of the connections between each node in the convolution operation of a backpropagation neural network.

[0016] The process involves deriving multiple channel images from a super-resolution model and then using multiple corresponding masks to derive the output image. This process of deriving the output image is the same as deriving multiple corresponding masks based on phase information.

[0017] The artificial intelligence super-resolution model obtained by applying the training method of the above-mentioned non-integer magnification artificial intelligence super-resolution model realizes a super-resolution method, in which an input image can be processed by the artificial intelligence super-resolution model to obtain an output image magnified by a non-integer magnification according to a non-integer magnification.

[0018] The publication also proposes a system applying the aforementioned artificial intelligence super-resolution model. The system's main circuit components include a circuit for running the super-resolution model on an input image, a memory for storing the input image, and an image magnification convolution operation circuit that determines the model parameters of the artificial intelligence super-resolution model based on a non-integer magnification and the position of the output pixels. Specifically, the system operates a non-integer magnification super-resolution method, which, based on the non-integer magnification, uses the artificial intelligence super-resolution model to produce an output image magnified by a non-integer magnification from the input image.

[0019] Preferably, the system implementation is applied to a special application integrated circuit in an audio-visual device.

[0020] Furthermore, the system can run the super-resolution method by reducing the operating frequency of the circuit running the super-resolution model to an appropriate scale.

[0021] Furthermore, the memory can be a first-in-first-out static random access memory. Furthermore, the system can enable the circuitry running the super-resolution model and the image magnification convolution operation circuitry to operate at the same operating frequency, and use this first-in-first-out static random access memory to run the super-resolution method.

[0022] The model parameters provided to the image magnification convolution operation circuit can be selected from a weight value library to determine the convolution weight values ​​used in the convolution operation of the artificial intelligence super-resolution model. The image magnification convolution operation circuit can be designed with a single convolutional layer or multiple convolutional layers.

[0023] Furthermore, the model parameters provided to the image magnification convolution operation circuit are multiple sets of convolution weight values ​​obtained through a hybrid operation.

[0024] To further understand the features and technical content of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are for reference and illustration only and are not intended to limit the present invention. Attached Figure Description

[0025] Figure 1 This displays a schematic diagram illustrating the operation of existing super-resolution algorithms;

[0026] Figure 2 This displays an example diagram of the existing super-resolution algorithm with integer magnification.

[0027] Figure 3 Show a timing diagram of the existing super-resolution algorithm being run;

[0028] Figure 4 This diagram illustrates an example of an existing super-resolution algorithm that implements non-integer scaling.

[0029] Figure 5 A schematic diagram illustrating an implementation example of a super-resolution computation method for non-integer multiple AI;

[0030] Figure 6 An example diagram showing a super-resolution computation method for non-integer multiple AI;

[0031] Figure 7 A flowchart illustrating an embodiment of a training method for a super-resolution model of artificial intelligence with non-integer scaling is shown; and

[0032] Figure 8 Show a system functional block diagram for implementing a super-resolution model that runs non-integer multiples of artificial intelligence.

[0033] Symbol Explanation

[0034] 101: Input Image

[0035] 103: Super-resolution machine learning model

[0036] 105: Output Image

[0037] 201: Input Image

[0038] 203: Super-resolution machine learning model

[0039] 205: Output Image

[0040] 211: Single-line, single-pixel image

[0041] 212: Single-line double-pixel image

[0042] 213: Double-line single-pixel image

[0043] 214: Double-line, double-pixel image

[0044] 301: Input image signal

[0045] 305: Output image signal

[0046] 401: Input Image

[0047] 403: Super-resolution machine learning model

[0048] 405: First Output Image

[0049] 407: Image Magnification Converter

[0050] 409: Second Output Image

[0051] 501: Input Image

[0052] 503: Super-resolution model

[0053] 505: Output Image

[0054] 511: Channel 1 Diagram

[0055] 512: Channel 2 Diagram

[0056] 513: The Nth 2 -1 Channel Chart

[0057] 514: The Nth 2 Channel diagram

[0058] 521: First Mask

[0059] 522: Second Mask

[0060] 523: The Nth 2 -1 mask

[0061] 524: The Nth 2 mask

[0062] 601: Input Image

[0063] 611: Channel 1 Diagram

[0064] 612: Channel 2 Diagram

[0065] 613: Channel 3 Diagram

[0066] 614: Channel 4 Diagram

[0067] 621: First Mask

[0068] 622: Second Mask

[0069] 623: Third Mask

[0070] 624: 4th Mask

[0071] 603: Composite Image

[0072] 605: Output Image

[0073] 801: Input Image

[0074] 803: Circuit for running super-resolution models

[0075] 805: Memory

[0076] 807: Image Magnification Convolution Operation Circuit

[0077] 809: Weight Value Library

[0078] 811: Phase Information

[0079] 813: Output Image

[0080] Steps S701 to S717: Training the super-resolution model Detailed Implementation

[0081] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of the present invention. Furthermore, the accompanying drawings of the present invention are for simple illustrative purposes only and are not depictions of actual dimensions; this is stated beforehand. The following embodiments will further describe the relevant technical content of the present invention in detail, but the disclosed content is not intended to limit the scope of protection of the present invention.

[0082] It should be understood that while terms such as "first," "second," and "third" may be used in this document to describe various components or signals, these components or signals should not be limited by these terms. These terms are primarily used to distinguish one component from another, or one signal from another. Furthermore, the term "or" as used herein should, as appropriate, include any combination of one or more of the associated listed items.

[0083] The publication discloses a training method, super-resolution method, and system for a super-resolution model, involving training a super-resolution model using a neural network-like system, a method for performing super-resolution using this model, and related hardware, enabling input images to achieve non-integer magnification of the image through a non-integer magnification artificial intelligence super-resolution model.

[0084] The image magnification is defined as applying a non-integer (mapping over integers) magnification (ratio = N / M) to an input image, where N and M are natural numbers. The method for training a super-resolution model of non-integer magnification artificial intelligence can be based on... Figure 5 The mask derived in the example is used for back propagation of a neural network to train a super-resolution model of non-integer multiple AI.

[0085] exist Figure 5 In the illustrated process, an input image (WxH, where W is the width and H is the height) 501 is first provided. The input image is then subjected to image magnification convolution calculation using a super-resolution model (SR model) 503. The magnification (N / M) is determined based on the input image 501. 2 Zhang WxH image, as shown in the attached image N 2 The images are represented by channels 1 (image 511), 2 (image 512), ..., N. 2 -1 Channel Diagram 513 and the Nth 2 Channel diagram 514.

[0086] Because the super-resolution calculation method of non-integer magnification artificial intelligence determines the final output image 505 at the magnification (N / M) before running, when N is obtained through the super-resolution model... 2Each channel image (WxH) needs to be reassembled (shuffled) into an output image 505 of N / M times WxH (N / Mx(WxH)), requiring a one-to-one mapping mask in between. According to an embodiment, the method can derive the mask corresponding to each channel image based on the phase information obtained from the magnification (N / M) and the position of the output pixels, as shown in the figure, including the first mask 521, the second mask 522, ..., the Nth mask. 2 -1 mask 523 and the Nth 2 Mask 524.

[0087] The above mask is designed pixel by pixel based on the input image 501. For example, the pixel in each channel image that corresponds to the output image 505 is marked as 1, and the other pixels that do not have a corresponding pixel are marked as 0, thus forming a mask for each channel image. Therefore, the output image 505 can be derived (inference) from the multiple channel images obtained from the super-resolution model through the mask design. The process of deriving the output image is the process of obtaining the mask based on the phase information of the channel images.

[0088] Furthermore, when establishing a super-resolution model with non-integer magnification, the phase information derived from the non-integer magnification and the position of the output pixels can be used to obtain a mask for the corresponding channel map (511, 512, 513, 514) to train a neural network, thereby establishing a new super-resolution model or updating an existing one. According to one embodiment, a backpropagation neural network (BPNN) can be introduced, and the mask designed according to requirements can be used to train the BPNN, thereby training the model parameters of the super-resolution model. See the embodiment for further details. Figure 7 The model parameters can refer to the weight values ​​of the connections between each node in the convolution operation of a backpropagation-type neural network. In the super-resolution calculation requirements of non-integer magnification artificial intelligence, the model parameters are the parameters in the convolution operation of image magnification. After training, a super-resolution model is obtained, which can be used to convert the input image 501 into the output image 505 according to the magnification requirement. Through the above-mentioned masking training, a super-resolution model for non-integer magnification can be implemented.

[0089] Figure 6 This diagram shows an example of a super-resolution calculus method that runs non-integer multiples of artificial intelligence. Figure 6Displaying a 2x2 (WxH) input image 601, a super-resolution algorithm based on non-integer magnification artificial intelligence is used to ultimately obtain a 3x3 output image 605. This involves determining multiple image channels based on a scaling factor of 2x, and determining the first channel image 611, the second channel image 612, the third channel image 613, and the fourth channel image 614 based on the target image formed by the 2x2 input image 601 and the non-integer magnification, i.e., the 3x3 output image 605. The corresponding mask is also derived based on the requirements of the 3x3 output image 605. In this example, the pixels ((0,0),(0,2),(2,0),(2,2)) of the first channel image 611 correspond to the first mask 621 as (1,0,0,0); the pixels ((0,1),(0,3),(2,1),(2,3)) of the second channel image 612 correspond to the second mask 622 as (1,1,0,0); the pixels ((1,0),(1,2),(3,0),(3,2)) of the third channel image 613 correspond to the third mask 623 as (1,0,1,0); and the pixels ((1,1),(1,3),(3,1),(3,3)) of the fourth channel image 614 correspond to the fourth mask 624 as (1,1,1,1). It can be seen that, under the requirement of reconstructing the output image 605, by referring to the image features of the output pixels and obtaining phase information from each channel image, multiple masks can be determined. A composite image 603 can be formed from multiple channel images. In this example, the pixels are labeled as (0,0),(0,1),(0,2),(0,3),(1,0),(1,1),(1,2),(1,3),(2,0),(2,1),(2,2),(2,3),(3,0),(3,1),(3,2),(3,3).

[0090] After applying the mapped mask, pixels (0,2), (1,2), (2,0), (2,1), (2,2), (2,3), and (3,2) in the combined image 603 can be masked (i.e., mask value 0). The remaining pixels with a mask value of 1 ((0,0), (0,1), (0,3), (1,0), (1,1), (1,3), (3,0), (3,1), and (3,3)) form a 3x3 output image 605. Using the above method, masks for various scale requirements can be obtained. These masks can be used for backpropagation in a neural network to train the model parameters in an artificial intelligence super-resolution model.

[0091] Figure 7 This document describes a method and implementation example for training a super-resolution model in a super-resolution algorithm for non-integer magnification artificial intelligence using the aforementioned masking.

[0092] The super-resolution model running in the above method is an artificial intelligence model based on neural network technology. In the step of training this super-resolution model with a large amount of image data, an input image is provided each time (step S701), and the expected magnification and the image quality threshold of the output image are set, which also determines the mask used in this operation.

[0093] The pixel values ​​of the input image are obtained, and the image features are acquired simultaneously (step S703). Then, based on the image features and the set magnification, multiple channel images are generated through a super-resolution model. This is a necessary process in executing the super-resolution algorithm (step S705). Next, based on the magnification and the position of the output pixels, phase information is obtained (step S707), and thus a mask corresponding to each channel image based on the magnification to be achieved is obtained in this operation (step S709). The method for generating the mask can be found in [reference needed]. Figure 5 The example image is shown below. After applying the corresponding mask to the channel image, iteratively updatable model parameters are applied, and a super-resolution algorithm is used to obtain a reconstructed image based on the set magnification (step S711). Next, the image is compared to the ideal value, i.e., the quality threshold of the output image (step S713), and a difference value is obtained. This difference value is used to evaluate the quality of the current mask and to train the artificial intelligence super-resolution model (step S715). After repeating the above steps, the iteration procedure is entered (step S717), repeatedly updating the model parameters of the artificial intelligence super-resolution model, and then applying them to step S711, i.e., using the updated model parameters to obtain the image using the super-resolution algorithm, ensuring that the output image meets the quality threshold set by the system.

[0094] Furthermore, all steps are repeated, different input images are input, and a large number of images and iterative procedures are used to converge the model parameters of the artificial intelligence super-resolution model. This also updates the weight values ​​of the connections between the input layer, hidden layer and output layer in the neural network, thus establishing an effective artificial intelligence super-resolution model.

[0095] The purpose of this process is to establish an artificial intelligence super-resolution model. The super-resolution method implemented through this model, based on a non-integer magnification, can directly generate an output image magnified by a non-integer magnification. The aforementioned artificial intelligence super-resolution model and related processes can also be applied to… Figure 8 Displayed in a super-resolution system.

[0096] According to the embodiment, the system uses hardware such as circuits in conjunction with intelligent algorithms to implement a super-resolution algorithm for non-integer multiple-rate artificial intelligence. Relevant hardware embodiments can be found in the following references. Figure 8The displayed system functional block diagram shows that each functional block can be implemented by circuits or hardware combined with software, and can ultimately be implemented as an application-specific integrated circuit (ASIC), which can be used in specific audio-visual devices, such as televisions, cameras and set-top boxes.

[0097] The system functionality for implementing the AI ​​super-resolution algorithm can be divided into three parts, mainly including the circuit 803 for proposing and running the super-resolution model (SR model), the memory 805, and the image magnification convolution operation circuit 807. The super-resolution model is achieved through... Figure 7 The AI ​​super-resolution model trained through the process operates at a frequency that may be the same as or different from the image magnification convolution operation circuit 807, running on an input image. The memory 805 can be implemented using a first-in-first-out (FIFO) static random access memory (SRAM), but other forms of memory are not excluded. The image magnification convolution operation circuit 807 determines the model parameters of the AI ​​super-resolution model based on a non-integer magnification and the image features of the input image 801, thereby achieving the purpose of non-integer magnification of the image. For example, phase information 811 is obtained based on the non-integer magnification (N / M) and the position of the output pixels. Convolution weights in the convolution operation can be selected from the weight value library 809 based on the phase information 811. The memory 805 stores the feature values ​​after the circuit 803, which runs the super-resolution model, has completed its operations.

[0098] Specifically, the aforementioned components can be implemented using logic circuits. The circuit 803 running the super-resolution model and the image magnification convolution operation circuit 807 can operate at the same or different (with a multiple relationship) clock frequencies, which can improve the system's operational flexibility and effectively save energy. When the system runs a non-integer magnification super-resolution method, there are two operating modes. One is to reduce the operating frequency of the circuit 803 running the super-resolution model to an appropriate ratio, which can effectively reduce power consumption while achieving the same super-resolution algorithm performance. The other is to run the circuit 803 running the super-resolution model and the image magnification convolution operation circuit 807 at the same operating frequency. The latter uses a first-in-first-out static random access memory, which can handle super-resolution algorithms at various magnifications to make more complete use of the artificial intelligence super-resolution model. This ensures that there is no circuit delay problem when performing convolution multiplication, thereby improving the performance of the super-resolution algorithm.

[0099] According to the embodiment, when the system is running, the circuit 803 that runs the super-resolution model on the input image 801 will input the input image 801 line by line into the memory 805 according to a non-integer magnification. At this time, a weight bank 809 is introduced. Based on the phase information 811 obtained from the non-integer magnification and the pixel position of the output image, the convolution weights corresponding to each pixel in the image are selected from the weight bank 809, which are the connections between the layers in the neural network. Therefore, it is not necessary to input the extra pixels that need to be obtained by masking as described above.

[0100] For example, taking a non-integer multiple (N / M = 7 / 5) as an example, it requires 7x7 (N 2 Each pixel in the output image 813 has corresponding phase information 811. Based on this phase information 811, a set of convolution weights matching the phase information 811 can be selected from the weight value library 809 to continue the operation of the image magnification convolution circuit 807, resulting in the final output image 813. This system design enables artificial intelligence super-resolution algorithms to process images magnified at non-integer magnification rates.

[0101] Furthermore, in one embodiment, the image magnification convolution operation circuit 807 can be designed as a single convolutional layer or multiple convolutional layers. That is, the image magnification convolution operation circuit 807 can obtain multiple layers of convolutional weight values ​​from the weight value library 809, and it is also possible to use other types of neural networks, such as residual networks.

[0102] Furthermore, the method of obtaining convolution weight values ​​from the weight value library 809 based on phase information 811 and providing them to the image magnification convolution operation circuit 807 is not intended to limit the applicability of the super-resolution method. Other derivative approaches can be used to increase flexibility or adjust different image qualities. For example, with a non-integer magnification of N / M, originally only one set of NxN convolution weight values ​​was needed, but the system can provide multiple sets (k sets) of NxN values. During the training of the artificial intelligence super-resolution model, a set of coefficients (c0, c1 to ck) corresponding to k sets of convolution weight values ​​can be given according to different application requirements. Finally, a set of convolution weight values ​​is obtained through blending (+) and then provided to the image magnification convolution operation circuit 807.

[0103] In summary, the training method, super-resolution method, and system for the super-resolution model described in the above embodiments utilize artificial intelligence learning techniques to train a new artificial intelligence super-resolution model by adding a mask designed based on non-integer magnification and output pixel information to an existing super-resolution model, thereby realizing a non-integer magnification super-resolution algorithm.

[0104] The content disclosed above is only a preferred and feasible embodiment of the present invention, and is not intended to limit the claims of the present invention. Therefore, all equivalent technical changes made based on the content of the present invention specification and drawings are included within the scope of the claims of the present invention.

Claims

1. A training method for a super-resolution model of non-integer multiple artificial intelligence, comprising: Provide an input image, set a magnification and an image quality threshold; Obtain the pixel values ​​of the input image and acquire the image features; Based on the image features of the input image and the magnification, multiple channel images are obtained through a super-resolution model; Based on this magnification, and by referring to the phase information obtained from the position of the output pixel, the mask corresponding to each channel image is obtained; After applying corresponding masks to the multiple channel images, a reconstructed output image is obtained; and Based on the image quality threshold, the multiple masks used to obtain the output image are evaluated to train an artificial intelligence super-resolution model. 2.The method of claim 1, wherein, By repeatedly following the steps of the training method for the super-resolution model of non-integer magnification artificial intelligence, the model parameters of the super-resolution model of non-integer magnification artificial intelligence are updated through an iterative procedure, so that the output image meets the image quality threshold.

3. The training method for a super-resolution model of non-integer multiplier artificial intelligence as described in claim 2, wherein the step of training the artificial intelligence super-resolution model further includes: By repeatedly inputting different input images and using a large number of images and iterative procedures, the model parameters of the artificial intelligence super-resolution model are converged. These model parameters are the weight values ​​of the connections between each node in a convolution operation of a backpropagation neural network.

4. The training method for a super-resolution model of non-integer multiplier artificial intelligence as described in claim 2, wherein the output image is derived from multiple channel images obtained from the super-resolution model through the corresponding multiple masks, and the process of deriving the output image is the process of obtaining the corresponding multiple masks based on the phase information.

5. The training method for the super-resolution model of non-integer multiplier artificial intelligence as described in claim 4, wherein in the design of the mask, the pixels of each channel image are marked with 1 corresponding to the pixels of the output image, and the remaining uncorresponding pixels are marked with 0, thus forming a mask corresponding to each channel image.

6. A super-resolution method, which uses an artificial intelligence super-resolution model obtained by training a non-integer magnification artificial intelligence super-resolution model as described in claim 1, to pass an input image through the artificial intelligence super-resolution model to obtain an output image magnified by a non-integer magnification according to a non-integer magnification.

7. A system applying an artificial intelligence super-resolution model, wherein the artificial intelligence super-resolution model is obtained through the training method of the non-integer multiplier artificial intelligence super-resolution model as described in claim 1, the system comprising: A circuit that runs a super-resolution model, which is used to run the AI ​​super-resolution model on an input image; A memory is used to store the feature values ​​of the circuit that runs the super-resolution model after it has completed its calculations. as well as An image magnification convolution operation circuit determines the model parameters of the artificial intelligence super-resolution model based on a non-integer magnification and the position of the output pixel. The system employs a non-integer magnification super-resolution method, which uses the artificial intelligence super-resolution model to generate an output image that has been magnified by a non-integer magnification based on the non-integer magnification of the input image.

8. The system of claim 7 that applies an artificial intelligence super-resolution model, wherein the system is implemented in a special application integrated circuit in an audio-visual device.

9. The system for applying an artificial intelligence super-resolution model as described in claim 8, wherein the system operates the super-resolution method by reducing the operating frequency of the circuitry running the super-resolution model to an appropriate scale.

10. The system for applying an artificial intelligence super-resolution model as described in claim 8, wherein the memory is a first-in-first-out static random access memory, the system enables the circuit running the super-resolution model and the image magnification convolution operation circuit to operate at the same operating frequency, and uses the first-in-first-out static random access memory to run the super-resolution method.

Images