Image super-resolution method based on deep learning

Abstract

The invention discloses an image super-resolution method based on deep learning, and relates to the technical field of image processing, and the method comprises the steps: carrying out the frequency domain denoising of a low-resolution image, and obtaining a no-noise low-resolution image; and carrying out position analysis on the non-noise low-resolution image. According to the method, firstly, the noise of the low-resolution image is removed through a frequency domain denoising method, the influence of the noise on the low-resolution image in the super-resolution process is avoided, the quality of the super-resolution image is improved, secondly, the pixel value of the low-resolution image is positioned, and after the super-resolution of the low-resolution image is completed, the super-resolution image is obtained. The method comprises the following steps: carrying out pixel verification on a super-resolution image through a pixel value of a low-resolution image, and finally, when pixel loss occurs in the super-resolution process of the low-resolution image, carrying out pixel value replacement on a coordinate of a pixel loss position after adjusting the pixel value of the low-resolution image with the same coordinate. The situation that partial areas of the super-resolution image are not coordinated is avoided, and the quality of the super-resolution image is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image super-resolution method based on deep learning. Background Art

[0002] In today's digital age, images, as essential vehicles for information dissemination and storage, play a crucial role in numerous fields, with their quality and resolution playing a crucial role. From security surveillance and medical imaging diagnostics to satellite remote sensing and consumer electronics, the demand for high-resolution images is growing. However, due to hardware limitations, bandwidth constraints, and complex shooting environments, the images actually captured are often low-resolution, significantly limiting their application value. In security surveillance, low-resolution images fail to clearly capture detailed features of objects, such as faces and license plates, creating significant challenges for incident tracking and crime investigations. In medical imaging diagnostics, low-resolution X-rays, CT scans, and other imaging techniques can cause doctors to miss tiny lesions, hindering early detection and accurate diagnosis, and ultimately delaying treatment. In satellite remote sensing, low-resolution images cannot accurately capture subtle changes in geographical features, making them inadequate for high-precision tasks such as resource exploration and environmental monitoring. In consumer electronics, users are increasingly demanding higher image quality for mobile phone photography and video playback, and low-resolution images fail to provide a satisfactory visual experience.

[0003] When existing low-resolution images are super-resolved, pixels may be lost, resulting in the super-resolved image having some areas that are not coordinated, thereby reducing the quality of the super-resolved image. Summary of the Invention

[0004] In order to solve the above technical problems, a deep learning-based image super-resolution method is provided. This technical solution solves the problems raised in the above background technology.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is: An image super-resolution method based on deep learning, comprising: Obtain low-resolution images, perform frequency domain denoising on the low-resolution images based on the intelligent analysis terminal, and obtain noise-free low-resolution images; Based on the intelligent analysis terminal, the position analysis of the noise-free low-resolution image is carried out to determine the pixel parameter coordinates; Based on the intelligent analysis terminal, a deep neural network model is constructed to perform super-resolution processing on the noise-free low-resolution image to obtain a super-resolution image; Based on the intelligent analysis terminal, the pixel parameter coordinates and the super-resolution image are compared and analyzed to determine whether the pixel value of the super-resolution image needs to be replaced.

[0006] Preferably, the step of acquiring a low-resolution image and performing frequency domain denoising on the low-resolution image based on an intelligent analysis terminal to acquire a noise-free low-resolution image specifically comprises the following steps: Based on the intelligent analysis terminal, the database system is processed for image extraction to obtain low-resolution images; Based on the fast Fourier transform algorithm, the low-resolution image is converted into the frequency domain to obtain the spectrum information of the low-resolution image; Based on the intelligent analysis terminal, the spectrum information of the low-resolution image is calculated and processed to obtain the amplitude spectrum and power spectrum of the low-resolution image; Based on the intelligent analysis terminal, the amplitude spectrum and power spectrum of the low-resolution image are calculated and processed to obtain a noise-free low-resolution image.

[0007] Preferably, the step of calculating and processing the amplitude spectrum and power spectrum of the low-resolution image based on the intelligent analysis terminal to obtain the noise-free low-resolution image specifically includes the following steps: Based on the intelligent analysis terminal, the amplitude spectrum and power spectrum of the low-resolution image are calculated and processed to obtain the power spectrum density of the low-resolution image; Get all noise power characteristics of the image; Based on all noise power features of the image, the power spectrum density of the low-resolution image is matched to determine the noise type of the low-resolution image; Designing a corresponding denoising filter based on the noise type of the low-resolution image, wherein the denoising filter is any one or more of a low-pass filter, a high-pass filter, and a band-pass filter; Based on the denoising filter, the spectrum information of the low-resolution image is denoised to obtain the spectrum information of the noise-free low-resolution image; Based on the inverse Fourier transform algorithm, the spectrum information of the noise-free low-resolution image is converted into time domain to obtain the noise-free low-resolution image.

[0008] Preferably, the method of performing position analysis on the noise-free low-resolution image based on the intelligent analysis terminal to determine the pixel parameter coordinates specifically includes the following steps: The lower left corner of the noise-free low-resolution image is set as the coordinate origin, the horizontal direction of the noise-free low-resolution image is set as the X-axis, and the vertical direction of the noise-free low-resolution image is set as the Y-axis to construct a rectangular coordinate system for pixel values; Put the noise-free low-resolution image into a rectangular coordinate system of pixel values; Based on the intelligent analysis terminal, pixel values of the noise-free low-resolution image in a rectangular coordinate system of pixel values are positioned to determine pixel parameter coordinates, wherein the specific form of the pixel parameter coordinates is (x coordinate, y coordinate, pixel value).

[0009] Preferably, the method of constructing a deep neural network model based on an intelligent analysis terminal and performing super-resolution processing on a noise-free low-resolution image to obtain a super-resolution image specifically includes the following steps: Perform multi-scale decomposition on the input low-resolution image to generate feature maps at multiple scales, and normalize the feature maps at each scale to obtain multi-scale feature maps; Construct a deep residual feature extraction network, which includes cascaded residual attention modules. Each residual attention module consists of a dilated convolutional layer, a channel attention layer, and a spatial attention layer to extract local and global features of multi-scale feature maps. Inputting the multi-scale feature map into the deep residual feature extraction network, dynamically weighting and fusing the multi-scale features through the cross-scale feature fusion module to generate a fused high-dimensional feature map; An adaptive upsampling module is used to improve the resolution of the fused high-dimensional feature map. The adaptive upsampling module generates a high-resolution feature map based on a learnable interpolation kernel and superimposes it with the high-frequency residual information of the low-resolution image. Constructing a multi-objective joint loss function, which includes pixel-level L1 loss, gradient consistency loss, and feature adversarial loss. The feature adversarial loss calculates the difference in feature distribution between the generated image and the real high-resolution image through a pre-trained discriminator network. Iteratively updating network parameters based on a dynamic optimization algorithm that adaptively adjusts the learning rate according to the gradient amplitude of the loss function in the current training batch until the network converges to obtain a deep neural network model; The noise-free low-resolution image is input into the deep neural network model for super-resolution processing to obtain a super-resolution image.

[0010] Preferably, the comparing and analyzing the pixel parameter coordinates and the super-resolution image based on the intelligent analysis terminal to determine whether pixel value replacement of the super-resolution image is required specifically includes the following steps: Acquire size information of the super-resolution image and size information of the noise-free low-resolution image, wherein the size information includes horizontal size information and vertical size information; Based on the intelligent analysis terminal, the size information of the super-resolution image and the size information of the noise-free low-resolution image are compared to obtain the comparison pixel parameter coordinates. The specific form of the comparison pixel parameter coordinates is (enlarge the x coordinate, enlarge the y coordinate, and adjust the pixel value); Based on the comparison of pixel parameter coordinates, the pixel values of the super-resolution image are verified to determine whether pixel value replacement of the super-resolution image is required.

[0011] Preferably, the step of performing comparison processing on the size information of the super-resolution image and the size information of the noise-free low-resolution image based on the intelligent analysis terminal to obtain the comparison pixel parameter coordinates specifically includes the following steps: Based on the intelligent analysis terminal, the horizontal size information of the super-resolution image and the horizontal size information of the noise-free low-resolution image, as well as the vertical size information of the super-resolution image and the vertical size information of the noise-free low-resolution image are calculated and processed respectively to obtain the horizontal magnification ratio of the image and the vertical magnification ratio of the image; Based on the horizontal magnification ratio of the image, the x coordinate of the pixel parameter coordinate is calculated and processed to obtain the magnified x coordinate; Based on the vertical magnification ratio of the image, the y coordinate of the pixel parameter coordinate is calculated and processed to obtain the magnified y coordinate; Based on the intelligent analysis terminal, the interpolation core of the upsampling module is read and processed to obtain the interpolation formula; Based on the interpolation formula, the pixel value of the pixel parameter coordinate is adjusted to obtain the adjusted pixel value; Based on the amplified x-coordinate, the amplified y-coordinate and the adjusted pixel value, the x-coordinate, the y-coordinate and the pixel value of the pixel parameter coordinate are replaced to obtain the compared pixel parameter coordinate.

[0012] Preferably, the verifying of the pixel values of the super-resolution image based on the comparison of pixel parameter coordinates to determine whether pixel value replacement of the super-resolution image is required specifically comprises the following steps: Placing the super-resolution image into a pixel value rectangular coordinate system to obtain pixel coordinates of the super-resolution image, wherein the specific form of the pixel coordinates of the super-resolution image is consistent with the specific form of the comparison pixel parameter coordinates; Matching the pixel coordinates of the super-resolution image according to the magnified x-coordinate and the magnified y-coordinate of the comparison pixel parameter coordinates to obtain the pixel value to be verified of the super-resolution image; Based on the intelligent analysis terminal, the pixel values to be verified and the pixel values of the comparison pixel parameter coordinates of the super-resolution image are analyzed and processed to determine whether the pixel values of the super-resolution image need to be replaced.

[0013] Preferably, the intelligent analysis terminal is used to analyze and process the pixel values to be verified and the adjusted pixel values of the comparison pixel parameter coordinates of the super-resolution image to determine whether pixel value replacement of the super-resolution image is required, and specifically comprises the following steps: Based on the intelligent analysis terminal, the pixel values to be verified in the super-resolution image and the adjusted pixel values of the comparison pixel parameter coordinates are judged and processed; If the pixel value to be verified of the super-resolution image is the same as the adjusted pixel value of the comparison pixel parameter coordinate, the super-resolution image has no pixel loss and there is no need to replace the pixel value of the super-resolution image; If the pixel value to be verified in the super-resolution image is different from the adjusted pixel value of the comparison pixel parameter coordinate, pixel loss occurs in the super-resolution image, and the pixel value to be verified in the super-resolution image is replaced by the adjusted pixel value of the comparison pixel parameter coordinate.

[0014] Furthermore, a deep learning-based image super-resolution system is proposed, which is used to implement the above-mentioned deep learning-based image super-resolution method, including: An intelligent analysis terminal is used to perform denoising, pixel value position analysis, super-resolution processing, and image pixel value verification on low-resolution images, and to determine whether pixel value replacement is required for super-resolution images; A database system, wherein the database system is used to store low-resolution images; An image denoising module, which performs denoising processing based on spectral information of the low-resolution image; A coordinate determination module, which determines the position of the pixel values of the noise-free low-resolution image according to a rectangular coordinate system of pixel values to obtain pixel parameter coordinates; A model building module, wherein the model building module is used to build a deep neural network model; A deep neural network model is used to perform super-resolution processing on the noise-free low-resolution image to obtain a super-resolution image; An image verification module verifies the pixel values of the super-resolution image according to the comparison pixel parameter coordinates to determine whether pixel value replacement of the super-resolution image is required.

[0015] Furthermore, a storage medium is proposed, on which a computer program is stored. When the computer program is called and run, it executes the above-mentioned deep learning-based image super-resolution method.

[0016] Compared with the existing technology, the present invention provides an image super-resolution method based on deep learning, which has the following beneficial effects: The present invention first removes noise from a low-resolution image through a frequency domain denoising method, thereby preventing the noise from affecting the low-resolution image during super-resolution, thereby improving the quality of the super-resolution image. Secondly, by locating the pixel values of the low-resolution image, after the low-resolution image completes super-resolution, pixel verification of the super-resolution image is performed through the pixel values of the low-resolution image. Finally, when pixel loss occurs in the low-resolution image during super-resolution, the pixel values of the low-resolution image with the same coordinates are adjusted, and then the pixel values of the coordinates of the pixel loss position are replaced, thereby avoiding the situation where some areas of the super-resolution image are not coordinated, thereby improving the quality of the super-resolution image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the process of steps S100-S400 in the image super-resolution method based on deep learning proposed by the present invention; Figure 2 This is a structural block diagram of the deep learning-based image super-resolution system proposed in this invention. DETAILED DESCRIPTION

[0018] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0019] Reference Figure 1 As shown, an image super-resolution method based on deep learning includes: S100: Acquire a low-resolution image, and perform frequency domain denoising on the low-resolution image based on an intelligent analysis terminal to obtain a noise-free low-resolution image; S200, based on the intelligent analysis terminal, performs position analysis on the noise-free low-resolution image to determine the pixel parameter coordinates; S300, based on the intelligent analysis terminal, builds a deep neural network model to perform super-resolution processing on the noise-free low-resolution image to obtain a super-resolution image; S400, based on the intelligent analysis terminal, performing comparative analysis on the pixel parameter coordinates and the super-resolution image to determine whether pixel value replacement of the super-resolution image is required; Those skilled in the art will understand that when a low-resolution image is super-resolved, pixel loss may occur. When pixel loss occurs, it will cause inconsistency in some areas of the super-resolution image, affecting the overall quality of the super-resolution image. Therefore, the low-resolution image is first denoised to avoid the influence of noise when the low-resolution image is super-resolved, thereby indirectly improving the quality of the super-resolution image. In addition, by constructing a rectangular coordinate system of pixel values to locate each pixel value in the low-resolution image, when the low-resolution image is super-resolved, pixel loss may occur. By adjusting the pixel values of the same coordinates to replace the pixel values at the positions where the pixels are lost in the super-resolution image, inconsistency in some areas of the super-resolution image is avoided, thereby improving the quality of the super-resolution image.

[0020] Example 1 Obtain a low-resolution image and perform frequency domain denoising on the low-resolution image based on the intelligent analysis terminal to obtain a noise-free low-resolution image. The specific steps include the following: S101. Based on the intelligent analysis terminal, perform image extraction processing on the database system to obtain a low-resolution image; S102. Performing frequency domain conversion on the low-resolution image based on a fast Fourier transform algorithm to obtain frequency spectrum information of the low-resolution image; S103, calculating and processing the frequency spectrum information of the low-resolution image based on the intelligent analysis terminal to obtain the amplitude spectrum and power spectrum of the low-resolution image; It can be understood that the spectrum information of the low-resolution image is analyzed to determine the spectrum function corresponding to the spectrum information of the low-resolution image. Then, the coefficients of the spectrum function are extracted to obtain the real and imaginary parts of the spectrum function. The amplitude spectrum is the sum of the square of the real part of the spectrum function and the square of the imaginary part of the spectrum function. The quadratic root calculation is performed on it to obtain the amplitude spectrum. The power spectrum is the square of the absolute value of the spectrum function, that is, the negative number of the spectrum function is first changed to a positive number, and then the power spectrum is obtained by squaring it. The position of the noise can be determined in the power spectrum. S104. Calculate and process the amplitude spectrum and power spectrum of the low-resolution image based on the intelligent analysis terminal to obtain a noise-free low-resolution image. In this embodiment, noise may exist in the low-resolution image. If the noise is not removed, some areas may appear blurred when the low-resolution image is super-resolutioned. Therefore, by performing frequency domain analysis on the low-resolution image, the noise type and location are determined and the noise is removed, thereby indirectly improving the quality of the super-resolution image.

[0021] Example 2 Based on the intelligent analysis terminal, the amplitude spectrum and power spectrum of the low-resolution image are calculated and processed to obtain a noise-free low-resolution image. The specific steps include the following: S1041. Calculate and process the amplitude spectrum and power spectrum of the low-resolution image based on the intelligent analysis terminal to obtain the power spectrum density of the low-resolution image. S1042. Obtain all noise power features of the image; It is understandable that noise also has energy. It cannot be observed in the time domain, but is displayed as energy in the frequency domain. Therefore, by calculating the power spectrum, amplitude spectrum, and power spectral density and analyzing their energy, the location and energy of the noise can be determined. S1043. Based on all noise power features of the image, matching processing is performed on the power spectrum density of the low-resolution image to determine the noise type of the low-resolution image. S1044. Design a corresponding denoising filter based on the noise type of the low-resolution image, where the denoising filter is any one or more of a low-pass filter, a high-pass filter, and a band-pass filter; S1045, performing denoising processing on the spectrum information of the low-resolution image based on a denoising filter to obtain spectrum information of the noise-free low-resolution image; S1046. Based on an inverse Fourier transform algorithm, perform time domain conversion on the spectrum information of the noise-free low-resolution image to obtain a noise-free low-resolution image; In this embodiment, image noise includes white noise, pink noise, blue noise, etc. The power spectral density of white noise is flat, indicating that the noise power is the same at all frequencies. If the power spectrum presents a flat (or nearly flat) distribution within a wide frequency band, it is usually a manifestation of white noise. The power spectral density of pink noise is relatively high in the low-frequency part and gradually decays in the high-frequency part. If the decay rate of the power spectrum roughly conforms to the 1 / f law, it is likely pink noise. The power spectrum of blue noise shows that the higher the frequency, the greater the power. Therefore, by analyzing the power spectral density, the noise type of the low-resolution image is determined, and then a corresponding filter is designed to remove the corresponding noise. For example, white noise can be removed by a low-pass filter or a band-pass filter.

[0022] Example 3 Based on the intelligent analysis terminal, the position analysis of the noise-free low-resolution image is performed to determine the pixel parameter coordinates, which specifically includes the following steps: S201, setting the lower left corner of the noise-free low-resolution image as the coordinate origin, setting the horizontal direction of the noise-free low-resolution image as the X-axis, and setting the vertical direction of the noise-free low-resolution image as the Y-axis, to construct a pixel value rectangular coordinate system; S202, placing the noise-free low-resolution image into a pixel value rectangular coordinate system; S203. Based on the intelligent analysis terminal, perform positioning processing on the pixel values of the noise-free low-resolution image in the pixel value rectangular coordinate system to determine pixel parameter coordinates, wherein the specific form of the pixel parameter coordinates is (x coordinate, y coordinate, pixel value); In this embodiment, in order to subsequently replace pixel values in areas where pixel loss occurs, the coordinates of each pixel value in the low-resolution image need to be recorded. The pixel values of the low-resolution image can only replace areas in the super-resolution image that belong to the low-resolution image. However, the size of the super-resolution image is different from the size of the low-resolution image. Therefore, the coordinates of each pixel value in the low-resolution image need to be recorded. Subsequently, based on the size change ratio, the coordinates in the super-resolution image corresponding to the coordinates of each pixel value in the low-resolution image are determined, and then pixel value analysis is performed to determine whether pixel value loss occurs.

[0023] Example 4 Based on the intelligent analysis terminal, a deep neural network model is constructed to perform super-resolution processing on the noise-free low-resolution image. The specific steps to obtain the super-resolution image include the following: S301, performing multi-scale decomposition on the input low-resolution image to generate feature maps at multiple scales, and normalizing the feature maps at each scale to obtain a multi-scale feature map; It can be understood that the multi-scale decomposition adopts the Laplacian pyramid decomposition method with 3-5 decomposition layers. The feature map of each layer is generated by Gaussian blurring and downsampling to generate the feature map of the next layer, and each layer of feature map is normalized by local contrast normalization. S302: construct a deep residual feature extraction network, wherein the network includes a cascade of residual attention modules, each of which is composed of a dilated convolutional layer, a channel attention layer, and a spatial attention layer, for extracting local and global features of a multi-scale feature map; It can be understood that the void ratio of the dilated convolution layer of the residual attention module is set to 2-4, the channel attention layer uses global average pooling and a fully connected layer to generate channel weights, and the spatial attention layer generates a spatial weight matrix through a convolution layer and multiplies it point by point with the input feature map; S303, inputting the multi-scale feature map into the deep residual feature extraction network, and performing dynamic weighted fusion on the multi-scale features through the cross-scale feature fusion module to generate a fused high-dimensional feature map; It can be understood that the cross-scale feature fusion module dynamically adjusts the contribution weights of features at different scales through a gating mechanism. The gating mechanism is implemented by a two-layer fully connected network, with the input being the mean and variance of the feature maps at each scale, and the output being the corresponding weight coefficient. S304, using an adaptive upsampling module to improve the resolution of the fused high-dimensional feature map, wherein the adaptive upsampling module generates a high-resolution feature map based on a learnable interpolation kernel and superimposes the high-frequency residual information of the low-resolution image; It can be understood that the learnable interpolation kernel is implemented through the deconvolution layer, the interpolation kernel size is 5×5, the parameters are initialized as bilinear interpolation, and are updated through backpropagation during training; high-frequency residual information is extracted from the low-resolution image through a high-pass filter; S305, constructing a multi-objective joint loss function, wherein the loss function includes pixel-level L1 loss, gradient consistency loss, and feature adversarial loss, wherein the feature adversarial loss calculates the feature distribution difference between the generated image and the real high-resolution image through a pre-trained discriminator network; It is understandable that the calculation of feature adversarial loss adopts a multi-scale discriminator structure. The discriminator network contains three parallel convolutional sub-networks, which discriminate different scale areas of the generated image respectively, and the weighted sum of the output results is used as the adversarial loss value; S306. Iteratively update the network parameters based on a dynamic optimization algorithm, wherein the dynamic optimization algorithm adaptively adjusts the learning rate according to the gradient amplitude of the loss function in the current training batch until the network converges to obtain a deep neural network model; It can be understood that the generated high-resolution image is subjected to non-local mean denoising, the similarity window size of the denoising is 7×7, the search window size is 21×21, and the similarity weight is calculated by the Gaussian kernel function; S307: Input the noise-free low-resolution image into a deep neural network model for super-resolution processing to obtain a super-resolution image.

[0024] Example 5 Based on the intelligent analysis terminal, the pixel parameter coordinates and the super-resolution image are compared and analyzed to determine whether the pixel value of the super-resolution image needs to be replaced. The specific steps include the following: S401, obtaining size information of a super-resolution image and size information of a noise-free low-resolution image, wherein the size information includes horizontal size information and vertical size information; S402: Based on the intelligent analysis terminal, the size information of the super-resolution image and the size information of the noise-free low-resolution image are compared to obtain comparison pixel parameter coordinates, wherein the comparison pixel parameter coordinates are specifically in the form of (enlarge x coordinate, enlarge y coordinate, adjust pixel value); S403: Verify the pixel values of the super-resolution image based on the compared pixel parameter coordinates to determine whether pixel value replacement of the super-resolution image is required; In this embodiment, if there is a low-resolution pixel surrounded by several pixels, the newly generated high-resolution pixel value will be interpolated based on the color values of these surrounding pixels. When the pixel value increases, the size of the image will change, and the coordinates of the pixel values will change. Therefore, by calculating the size information of the super-resolution image and the size information of the noise-free low-resolution image, the comparison pixel parameter coordinates are determined, and then the pixel values of the super-resolution image are verified based on the adjusted pixel values of the comparison pixel parameter coordinates to determine whether the pixel values of the super-resolution image need to be replaced.

[0025] Example 6 Based on the intelligent analysis terminal, the size information of the super-resolution image and the size information of the noise-free low-resolution image are compared and the coordinates of the comparison pixel parameters are obtained, which specifically includes the following steps: S4021. Based on the intelligent analysis terminal, respectively calculate and process the horizontal size information of the super-resolution image and the horizontal size information of the noise-free low-resolution image, and the vertical size information of the super-resolution image and the vertical size information of the noise-free low-resolution image to obtain a horizontal magnification ratio of the image and a vertical magnification ratio of the image; S4022. Calculate and process the x-coordinate of the pixel parameter coordinate based on the horizontal magnification ratio of the image to obtain the magnified x-coordinate; S4023. Calculate the y coordinate of the pixel parameter coordinate based on the vertical magnification ratio of the image to obtain the magnified y coordinate; S4024. Based on the intelligent analysis terminal, read and process the interpolation core of the upsampling module to obtain an interpolation formula; S4025. Adjust the pixel value of the pixel parameter coordinate based on the interpolation formula to obtain an adjusted pixel value; S4026 , based on the amplified x-coordinate, the amplified y-coordinate, and the adjusted pixel value, the x-coordinate, the y-coordinate, and the pixel value of the pixel parameter coordinate are replaced to obtain a comparison pixel parameter coordinate; In this embodiment, as the pixel values increase, the image size changes. Therefore, the horizontal size information of the super-resolution image, the horizontal size information of the noise-free low-resolution image, and the vertical size information of the super-resolution image, as well as the vertical size information of the noise-free low-resolution image, are calculated and processed to obtain the horizontal and vertical image magnification ratios. Then, the x- and y-coordinates of the pixel parameter coordinates are magnified according to the image magnification ratios. The magnified x- and y-coordinates are the coordinates of the pixel values of the original low-resolution image in the super-resolution image. However, to adapt to the super-resolution pixels, the pixel values in the low-resolution image will also change. However, the changes in the pixel values in the low-resolution image are adjusted using an interpolation formula. Therefore, by reading data from the interpolation kernel of the upsampling module and determining the interpolation formula, the low-resolution pixel values are calculated according to the interpolation formula to obtain adjusted pixel values. Subsequently, the super-resolution image can be verified based on the adjusted pixel values to determine whether there are missing pixels in the super-resolution image.

[0026] Example 7 Based on the comparison of pixel parameter coordinates, the pixel values of the super-resolution image are verified to determine whether the pixel values of the super-resolution image need to be replaced. Specifically, the steps include: S4031, placing the super-resolution image into a pixel value rectangular coordinate system, and obtaining pixel coordinates of the super-resolution image, wherein the specific form of the pixel coordinates of the super-resolution image is consistent with the specific form of the comparison pixel parameter coordinates; S4032: performing matching processing on the pixel coordinates of the super-resolution image according to the magnified x-coordinate and the magnified y-coordinate of the comparison pixel parameter coordinates to obtain the pixel value to be verified of the super-resolution image; S4033: Analyze and process the pixel values to be verified and the adjusted pixel values of the comparison pixel parameter coordinates of the super-resolution image based on the intelligent analysis terminal to determine whether pixel value replacement of the super-resolution image is required; In this embodiment, since the size of a low-resolution image changes when it undergoes super-resolution, the coordinates of the pixel values of the original low-resolution image will change accordingly. Therefore, it is necessary to determine the coordinates after magnification in order to determine the position of the pixel values of the original low-resolution image in the super-resolution image and to perform subsequent pixel value verification.

[0027] Example 8 Based on the intelligent analysis terminal, the pixel values to be verified and the adjusted pixel values of the comparison pixel parameter coordinates of the super-resolution image are analyzed and processed to determine whether the pixel values of the super-resolution image need to be replaced. The specific steps include the following: S40331. Based on the intelligent analysis terminal, determine and process the pixel value to be verified of the super-resolution image and the adjusted pixel value of the comparison pixel parameter coordinate; S40332: If the pixel value to be verified of the super-resolution image is the same as the adjusted pixel value of the comparison pixel parameter coordinate, the super-resolution image has no pixel loss, and there is no need to replace the pixel value of the super-resolution image. S40333: If the pixel value to be verified in the super-resolution image is different from the adjusted pixel value of the comparison pixel parameter coordinate, pixel loss occurs in the super-resolution image, and the pixel value to be verified in the super-resolution image is replaced by the adjusted pixel value of the comparison pixel parameter coordinate; In this embodiment, when a low-resolution image is super-resolved, missing pixels may occur, which may cause incoordination in some areas of the super-resolution image. Therefore, the pixel values of the low-resolution image are adjusted through an interpolation formula to obtain pixel value information of the super-resolution image. When the adjusted pixel value is inconsistent with the pixel value to be verified of the super-resolution image, it means that missing pixels have occurred in the low-resolution image during super-resolution. Therefore, in order to improve the quality of the super-resolution image, the pixel value to be verified is replaced with the adjusted pixel value. It can be understood that the adjusted pixel value is the pixel value of the low-resolution image after super-resolution. If the pixel value after super-resolution is inconsistent with the adjusted pixel value, it means that pixel loss has occurred during super-resolution. In order to make each area of the super-resolution image more coordinated, it is replaced.

[0028] Reference Figure 2 As shown, a deep learning-based image super-resolution system is used to implement the above-mentioned deep learning-based image super-resolution method, including: An intelligent analysis terminal is used to perform denoising, pixel value position analysis, super-resolution processing, and image pixel value verification on low-resolution images, and to determine whether pixel value replacement is required for super-resolution images; A database system, wherein the database system is used to store low-resolution images; An image denoising module, which performs denoising processing based on spectral information of the low-resolution image; A coordinate determination module, which determines the position of the pixel values of the noise-free low-resolution image according to a rectangular coordinate system of pixel values to obtain pixel parameter coordinates; A model building module, wherein the model building module is used to build a deep neural network model; A deep neural network model is used to perform super-resolution processing on the noise-free low-resolution image to obtain a super-resolution image; An image verification module verifies the pixel values of the super-resolution image according to the comparison pixel parameter coordinates to determine whether pixel value replacement of the super-resolution image is required.

[0029] Furthermore, a storage medium is proposed, on which a computer program is stored. When the computer program is called and run, it executes an image super-resolution method based on deep learning as described above, wherein the storage medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; an optical medium, such as a DVD; or a semiconductor medium, such as a solid-state drive (SSD).

[0030] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. An image super-resolution method based on deep learning, characterized in that: include: Obtain low-resolution images, perform frequency domain denoising on the low-resolution images based on the intelligent analysis terminal, and obtain noise-free low-resolution images; Based on the intelligent analysis terminal, the position analysis of the noise-free low-resolution image is carried out to determine the pixel parameter coordinates; Based on the intelligent analysis terminal, a deep neural network model is constructed to perform super-resolution processing on the noise-free low-resolution image to obtain a super-resolution image; Based on the intelligent analysis terminal, the pixel parameter coordinates and the super-resolution image are compared and analyzed to determine whether the pixel value of the super-resolution image needs to be replaced.

2. The image super-resolution method based on deep learning according to claim 1, characterized in that: The method of obtaining a low-resolution image and performing frequency domain denoising on the low-resolution image based on the intelligent analysis terminal to obtain a noise-free low-resolution image specifically includes the following steps: Based on the intelligent analysis terminal, the database system is processed for image extraction to obtain low-resolution images; Based on the fast Fourier transform algorithm, the low-resolution image is converted into the frequency domain to obtain the spectrum information of the low-resolution image; Based on the intelligent analysis terminal, the spectrum information of the low-resolution image is calculated and processed to obtain the amplitude spectrum and power spectrum of the low-resolution image; Based on the intelligent analysis terminal, the amplitude spectrum and power spectrum of the low-resolution image are calculated and processed to obtain a noise-free low-resolution image.

3. The image super-resolution method based on deep learning according to claim 2, characterized in that: The method of calculating and processing the amplitude spectrum and power spectrum of the low-resolution image based on the intelligent analysis terminal to obtain a noise-free low-resolution image specifically includes the following steps: Based on the intelligent analysis terminal, the amplitude spectrum and power spectrum of the low-resolution image are calculated and processed to obtain the power spectrum density of the low-resolution image; Get all noise power characteristics of the image; Based on all noise power features of the image, the power spectrum density of the low-resolution image is matched to determine the noise type of the low-resolution image; Designing a corresponding denoising filter based on the noise type of the low-resolution image, wherein the denoising filter is any one or more of a low-pass filter, a high-pass filter, and a band-pass filter; Based on the denoising filter, the spectrum information of the low-resolution image is denoised to obtain the spectrum information of the noise-free low-resolution image; Based on the inverse Fourier transform algorithm, the spectrum information of the noise-free low-resolution image is converted into time domain to obtain the noise-free low-resolution image.

4. The image super-resolution method based on deep learning according to claim 1, characterized in that: The method of performing position analysis on the noise-free low-resolution image based on the intelligent analysis terminal to determine the pixel parameter coordinates specifically includes the following steps: The lower left corner of the noise-free low-resolution image is set as the coordinate origin, the horizontal direction of the noise-free low-resolution image is set as the X-axis, and the vertical direction of the noise-free low-resolution image is set as the Y-axis to construct a rectangular coordinate system for pixel values; Put the noise-free low-resolution image into a rectangular coordinate system of pixel values; Based on the intelligent analysis terminal, pixel values of the noise-free low-resolution image in a rectangular coordinate system of pixel values are positioned to determine pixel parameter coordinates, wherein the specific form of the pixel parameter coordinates is (x coordinate, y coordinate, pixel value).

5. The image super-resolution method based on deep learning according to claim 1, characterized in that: The method of constructing a deep neural network model based on the intelligent analysis terminal and performing super-resolution processing on the noise-free low-resolution image to obtain the super-resolution image specifically includes the following steps: Perform multi-scale decomposition on the input low-resolution image to generate feature maps at multiple scales, and normalize the feature maps at each scale to obtain multi-scale feature maps; Construct a deep residual feature extraction network, which includes cascaded residual attention modules. Each residual attention module consists of a dilated convolutional layer, a channel attention layer, and a spatial attention layer to extract local and global features of multi-scale feature maps. Inputting the multi-scale feature map into the deep residual feature extraction network, dynamically weighting and fusing the multi-scale features through the cross-scale feature fusion module to generate a fused high-dimensional feature map; An adaptive upsampling module is used to improve the resolution of the fused high-dimensional feature map. The adaptive upsampling module generates a high-resolution feature map based on a learnable interpolation kernel and superimposes it with the high-frequency residual information of the low-resolution image. Constructing a multi-objective joint loss function, which includes pixel-level L1 loss, gradient consistency loss, and feature adversarial loss. The feature adversarial loss calculates the difference in feature distribution between the generated image and the real high-resolution image through a pre-trained discriminator network. Iteratively updating network parameters based on a dynamic optimization algorithm that adaptively adjusts the learning rate according to the gradient amplitude of the loss function in the current training batch until the network converges to obtain a deep neural network model; The noise-free low-resolution image is input into the deep neural network model for super-resolution processing to obtain a super-resolution image.

6. The image super-resolution method based on deep learning according to claim 1, characterized in that: The method of comparing and analyzing the pixel parameter coordinates and the super-resolution image based on the intelligent analysis terminal to determine whether pixel value replacement of the super-resolution image is required specifically includes the following steps: Acquire size information of the super-resolution image and size information of the noise-free low-resolution image, wherein the size information includes horizontal size information and vertical size information; Based on the intelligent analysis terminal, the size information of the super-resolution image and the size information of the noise-free low-resolution image are compared to obtain the comparison pixel parameter coordinates. The specific form of the comparison pixel parameter coordinates is (enlarge the x coordinate, enlarge the y coordinate, and adjust the pixel value); Based on the comparison of pixel parameter coordinates, the pixel values of the super-resolution image are verified to determine whether pixel value replacement of the super-resolution image is required.

7. The image super-resolution method based on deep learning according to claim 6, characterized in that: The intelligent analysis terminal is used to compare the size information of the super-resolution image and the size information of the noise-free low-resolution image to obtain the comparison pixel parameter coordinates, which specifically includes the following steps: Based on the intelligent analysis terminal, the horizontal size information of the super-resolution image and the horizontal size information of the noise-free low-resolution image, as well as the vertical size information of the super-resolution image and the vertical size information of the noise-free low-resolution image are calculated and processed respectively to obtain the horizontal magnification ratio of the image and the vertical magnification ratio of the image; Based on the horizontal magnification ratio of the image, the x coordinate of the pixel parameter coordinate is calculated and processed to obtain the magnified x coordinate; Based on the vertical magnification ratio of the image, the y coordinate of the pixel parameter coordinate is calculated and processed to obtain the magnified y coordinate; Based on the intelligent analysis terminal, the interpolation core of the upsampling module is read and processed to obtain the interpolation formula; Based on the interpolation formula, the pixel value of the pixel parameter coordinate is adjusted to obtain the adjusted pixel value; Based on the amplified x-coordinate, the amplified y-coordinate and the adjusted pixel value, the x-coordinate, the y-coordinate and the pixel value of the pixel parameter coordinate are replaced to obtain the compared pixel parameter coordinate.

8. The image super-resolution method based on deep learning according to claim 6, characterized in that: The verification process of the pixel values of the super-resolution image based on the comparison of pixel parameter coordinates to determine whether the pixel values of the super-resolution image need to be replaced specifically includes the following steps: Placing the super-resolution image into a pixel value rectangular coordinate system to obtain pixel coordinates of the super-resolution image, wherein the specific form of the pixel coordinates of the super-resolution image is consistent with the specific form of the comparison pixel parameter coordinates; Matching the pixel coordinates of the super-resolution image according to the magnified x-coordinate and the magnified y-coordinate of the comparison pixel parameter coordinates to obtain the pixel value to be verified of the super-resolution image; Based on the intelligent analysis terminal, the pixel values to be verified of the super-resolution image and the adjusted pixel values of the comparison pixel parameter coordinates are analyzed and processed to determine whether the pixel values of the super-resolution image need to be replaced.

9. The image super-resolution method based on deep learning according to claim 8, characterized in that: The intelligent analysis terminal analyzes and processes the pixel values to be verified and the adjusted pixel values of the comparison pixel parameter coordinates of the super-resolution image to determine whether pixel value replacement of the super-resolution image is required, specifically including the following steps: Based on the intelligent analysis terminal, the pixel values to be verified in the super-resolution image and the adjusted pixel values of the comparison pixel parameter coordinates are judged and processed; If the pixel value to be verified of the super-resolution image is the same as the adjusted pixel value of the comparison pixel parameter coordinate, the super-resolution image has no pixel loss and there is no need to replace the pixel value of the super-resolution image; If the pixel value to be verified in the super-resolution image is different from the adjusted pixel value of the comparison pixel parameter coordinate, pixel loss occurs in the super-resolution image, and the pixel value to be verified in the super-resolution image is replaced by the adjusted pixel value of the comparison pixel parameter coordinate.

10. A deep learning-based image super-resolution system, used to implement the deep learning-based image super-resolution method according to any one of claims 1 to 9, characterized in that: include: An intelligent analysis terminal is used to perform denoising, pixel value position analysis, super-resolution processing, and image pixel value verification on low-resolution images, and to determine whether pixel value replacement is required for super-resolution images; A database system, wherein the database system is used to store low-resolution images; An image denoising module, which performs denoising processing based on spectral information of the low-resolution image; A coordinate determination module, which determines the position of the pixel values of the noise-free low-resolution image according to a rectangular coordinate system of pixel values to obtain pixel parameter coordinates; A model building module, wherein the model building module is used to build a deep neural network model; A deep neural network model is used to perform super-resolution processing on the noise-free low-resolution image to obtain a super-resolution image; An image verification module verifies the pixel values of the super-resolution image according to the comparison pixel parameter coordinates to determine whether pixel value replacement of the super-resolution image is required.

Images