Image processing methods, apparatus, devices and storage media

Abstract

This application discloses an image processing method, apparatus, device, and storage medium, which can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, and vehicle networking. The method includes: acquiring an image to be processed with a resolution less than a preset threshold; performing resolution enhancement processing on the image to be processed based on an image processing model to obtain a target image; wherein, the training method of the image processing model includes: decomposing the initial sample shallow features of the first sample image into at least two target sample shallow features based on a preset model; performing resolution enhancement training on the sample deep features based on the preset model to adjust the parameters of the preset model until the resolution loss value of the preset model is less than a preset value; and using the preset model corresponding to the current model parameters as the image processing model. This application improves the clarity of the target image.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of image processing technology, the acquisition or detection of important information data through image capture has been widely applied in various industries. High-resolution images can often reflect the information in an image more clearly and accurately. Image super-resolution aims to restore low-resolution images, making them contain more detail and improving image clarity. Currently, image super-resolution techniques are mainly divided into interpolation-based methods and reconstruction-based methods. Interpolation-based methods assume that the gray values ​​of an image are continuously changing and can determine the gray value of the target pixel using the gray values ​​of surrounding pixels. Interpolation-based methods have high reconstruction efficiency, but they usually ignore more high-frequency details, resulting in poor reconstructed image quality. Reconstruction-based methods transform the super-resolution problem into a loss function optimization problem, utilizing the local smoothing characteristics of the image, the non-negative nature of pixel gray values, and finite energy as regularization terms. Although reconstruction-based methods can produce sharpened boundaries, they may miss high-frequency information when the magnification is too large. Summary of the Invention

[0003] This application provides an image processing method, apparatus, device, and storage medium that can accurately identify detailed information in the image to be processed and improve the clarity of the target image.

[0004] On one hand, this application provides an image processing method, the method comprising:

[0005] Acquire images to be processed with a resolution smaller than a preset threshold;

[0006] The image to be processed is subjected to resolution enhancement processing based on an image processing model to obtain a target image; the resolution of the target image is greater than the resolution of the image to be processed.

[0007] The training method for the image processing model includes:

[0008] Obtain a first sample image and a second sample image; the first sample image is obtained by reducing the resolution of the second sample image.

[0009] Based on a preset model, the initial shallow features of the first sample image are decomposed into at least two shallow features of the target sample.

[0010] Based on the preset model, the deep features of the sample are trained to enhance resolution, so as to adjust the parameters of the preset model until the resolution loss value of the preset model is less than the preset value; the deep features of the sample are obtained by extracting deep features from the shallow features of the at least two target samples;

[0011] The preset model corresponding to the current model parameters is used as the image processing model; the current model parameters are the model parameters when the resolution loss value is less than the preset value.

[0012] On the other hand, an image processing apparatus is provided, the apparatus comprising:

[0013] The image acquisition module is used to acquire images with a resolution less than a preset threshold.

[0014] An image processing module is used to perform resolution enhancement processing on the image to be processed based on an image processing model to obtain a target image; the resolution of the target image is greater than the resolution of the image to be processed.

[0015] A sample image acquisition module is used to acquire a first sample image and a second sample image; the first sample image is obtained by reducing the resolution of the second sample image; the resolution of the second sample image is greater than the preset threshold.

[0016] The sample shallow feature decomposition module is used to decompose the initial sample shallow features of the first sample image into at least two target sample shallow features based on a preset model.

[0017] The training module is used to perform resolution enhancement training on the deep features of the samples based on the preset model, so as to adjust the parameters of the preset model until the resolution loss value of the preset model is less than a preset value; the deep features of the samples are obtained by extracting deep features from the shallow features of the at least two target samples;

[0018] The model determination module is used to use the preset model corresponding to the current model parameters as the image processing model; the current model parameters are the model parameters when the resolution loss value is less than the preset value.

[0019] On the other hand, an image processing device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing method as described above.

[0020] On the other hand, a computer storage medium is provided that stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the image processing method described above.

[0021] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing method as described above.

[0022] The image processing method, apparatus, device, and storage medium provided in this application have the following technical effects:

[0023] This application acquires an image to be processed with a resolution less than a preset threshold; performs resolution enhancement processing on the image to be processed based on an image processing model to obtain a target image; wherein, the training method of the image processing model includes: acquiring a first sample image and a second sample image; decomposing the initial sample shallow features of the first sample image into at least two target sample shallow features based on a preset model; performing resolution enhancement training on the sample deep features based on the preset model to adjust the parameters of the preset model until the resolution loss value of the preset model is less than the preset value; the sample deep features are obtained by extracting deep features from the at least two target sample shallow features; and using the preset model corresponding to the current model parameters as the image processing model; the current model parameters are the model parameters when the resolution loss value is less than the preset value. In the model training process, this application decomposes the shallow features extracted from the image and then extracts deep features based on the decomposed shallow features for model training, thereby reducing the model parameters and the computational load during model training while ensuring model accuracy, and improving the model training speed; and the obtained image processing model can accurately identify the detailed information in the image to be processed, improving the clarity of the target image. Attached Figure Description

[0024] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of an image processing system provided in an embodiment of this application;

[0026] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0027] Figure 3 This is a flowchart illustrating a method for enhancing the resolution of deep features of a sample based on the preset model, as provided in an embodiment of this application.

[0028] Figure 4 This is a flowchart illustrating a method provided in this application for extracting deep features from shallow features of target samples corresponding to each enhanced dense connection block to obtain deep features of at least two samples, based on each enhanced dense connection block.

[0029] Figure 5 This is a flowchart illustrating a method provided in this application for obtaining at least one target deep feature by inputting the first deep feature and the shallow feature of the target sample corresponding to at least one target enhancement dense connection block into the at least one target enhancement dense connection block.

[0030] Figure 6 This is a flowchart illustrating the method for performing resolution enhancement training on the sample fusion features based on the preset model provided in this application embodiment;

[0031] Figure 7 This is a schematic diagram of the structure of a MEDCN network provided in an embodiment of this application;

[0032] Figure 8 This is a schematic diagram of the structure of an EDCB network provided in an embodiment of this application;

[0033] Figure 9 This is a schematic diagram of an image to be processed provided in an embodiment of this application;

[0034] Figure 10 These are schematic diagrams of high-resolution images obtained using different methods, provided in embodiments of this application.

[0035] Figure 11 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0036] Figure 12 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0038] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0039] Image super-resolution: recovering a high-resolution image from a low-resolution image or image sequence.

[0040] Convolutional Neural Networks (CNNs) are a class of feedforward neural networks that include convolutional computations and have a deep structure. They are one of the representative algorithms of deep learning.

[0041] Vanishing gradient: In neural networks, the learning rate of the preceding hidden layers is lower than that of the subsequent hidden layers. In other words, as the number of hidden layers increases, the accuracy decreases. This phenomenon is called vanishing gradient.

[0042] PSNR: Peak Signal to Noise Ratio, is an objective standard for evaluating images.

[0043] SSIM: Structural Similarity, is a metric that measures the similarity between two images.

[0044] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0045] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0046] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning. The solutions provided in this application's embodiments relate to computer vision technology and machine learning technologies within artificial intelligence.

[0047] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0048] Big data refers to data sets that cannot be captured, managed, and processed within a certain timeframe using conventional software tools. It represents massive, rapidly growing, and diverse information assets that require new processing models to achieve stronger decision-making, insightful discovery, and process optimization capabilities. With the advent of the cloud era, big data has attracted increasing attention. Big data requires specialized technologies to effectively process large amounts of data within a tolerable timeframe. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems. The solution provided in this application relates to big data in cloud technology. By performing super-resolution processing on images, high-resolution images are obtained, which can then be used for object recognition, classification, and accurate recommendations within the images.

[0049] Intelligent transportation fully utilizes next-generation information technologies such as the Internet of Things, spatial sensing, cloud computing, and mobile internet across the entire transportation sector. It comprehensively applies theories and tools from transportation science, systems methods, artificial intelligence, and knowledge mining. With the goals of comprehensive perception, deep integration, proactive service, and scientific decision-making, it builds a real-time dynamic information service system, deeply mines transportation-related data, forms problem analysis models, and enhances the industry's ability to optimize resource allocation, improve public decision-making capabilities, enhance industry management capabilities, and improve public service capabilities. This promotes safer, more efficient, more convenient, more economical, more environmentally friendly, and more comfortable operation and development of transportation, and drives the transformation and upgrading of transportation-related industries.

[0050] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0051] Please see Figure 1 , Figure 1 This is a schematic diagram of an image processing system provided in an embodiment of this application, such as... Figure 1 As shown, the image processing system may include at least server 01 and client 02.

[0052] Specifically, in this embodiment, server 01 may include a standalone server, a distributed server, or a server cluster composed of multiple servers. It may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 01 may include a network communication unit, a processor, and a memory, etc. Specifically, server 01 can be used to convert low-resolution images to be processed into high-resolution target images.

[0053] Specifically, in this embodiment, the client 02 may include physical devices such as smartphones, desktop computers, tablets, laptops, digital assistants, smart wearable devices, smart speakers, in-vehicle terminals, and smart TVs. It may also include software running on the physical device, such as web pages provided to users by service providers, or applications provided by those service providers. Specifically, the client 02 can be used to view the target image corresponding to the image to be processed online.

[0054] The following describes an image processing method according to this application, which is an image super-resolution method. Figure 2 This is a flowchart illustrating an image processing method provided in an embodiment of this application. This specification provides the operational steps of the method as described in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or accompanying drawings. Specifically, as follows... Figure 2 As shown, the method may include:

[0055] S201: Obtain images to be processed with a resolution less than a preset threshold.

[0056] In this embodiment of the application, the image to be processed with a resolution less than a preset threshold is a low-resolution image.

[0057] S203: Perform resolution enhancement processing on the image to be processed based on the image processing model to obtain a target image; the resolution of the target image is greater than the resolution of the image to be processed.

[0058] In this embodiment of the application, the target image is a high-resolution image corresponding to the image to be processed; clarifying the blurred image (the image to be processed) can be used as a preprocessing method for image classification and image recognition, effectively improving the accuracy of image classification and image recognition.

[0059] In this embodiment of the application, the image to be processed can be input into an image processing model for resolution enhancement processing to obtain a target image; the image processing model can be a multi-channel enhanced dense connection network (MEDCN), which can include a target shallow feature extraction network, a target shallow feature decomposition network, a target deep feature extraction network, a target feature fusion network, and a target sub-pixel convolutional network.

[0060] In this embodiment of the application, the step of performing resolution enhancement processing on the image to be processed based on the image processing model to obtain the target image may include:

[0061] The shallow features of the image to be processed are extracted based on the target shallow feature extraction network to obtain the first shallow feature;

[0062] In this embodiment of the application, the shallow feature extraction network may consist of a first number of convolutional layers, and the first shallow feature may be a feature of a first dimension, such as a 64-dimensional shallow feature.

[0063] Based on the target shallow feature decomposition network, the first shallow feature is decomposed into at least two second shallow features;

[0064] In the embodiments of this application, the first shallow feature can be decomposed into at least two second shallow features according to the dimensions; for example, the 64-dimensional shallow feature can be decomposed into four groups of 16-dimensional features to obtain four second shallow features.

[0065] Based on the target deep feature extraction network, the target deep features of the image to be processed are extracted from the at least two second shallow features;

[0066] In this embodiment of the application, the target deep feature extraction network may include a fixed number of Enhanced Dense Connection Blocks (EDCBs); each EDCB is densely connected to the others; and each EDCB is composed of multiple different convolutional networks.

[0067] The training method for the image processing model includes:

[0068] S301: Obtain a first sample image and a second sample image; the first sample image is obtained by reducing the resolution of the second sample image.

[0069] In this embodiment, the resolution of the second sample image is greater than the preset threshold; the first sample image is a low-resolution image, and the second sample image is a high-resolution image corresponding to the first sample image. The first and second sample images can be constructed into a sample image pair before model training.

[0070] In this embodiment, DIV2K data can be used as training sample images. DIV2K data is a super-resolution dataset containing 800 2K resolution training images, 100 validation images, and 100 test images. The 800 training images can be selected as training sample images. The training images include high-resolution images and corresponding low-resolution images. HR / LR image pairs can be constructed from the training images in the DIV2K data to train a preset model, resulting in an image processing model. Alternatively, bicubic interpolation downsampling can be used to process the high-resolution images in the training images. The downsampling factors can be selected as 2, 3, and 4 respectively to obtain low-resolution images with a preset resolution, and then sample image pairs can be constructed for model training. Bicubic interpolation is also known as cubic convolution interpolation. Cubic convolution interpolation is a more complex interpolation method. This algorithm uses the gray values ​​of 16 points surrounding the sampling point for cubic interpolation, considering not only the gray value influence of the four directly adjacent points but also the influence of the gray value change rate between neighboring points.

[0071] S303: Based on a preset model, the initial shallow features of the first sample image are decomposed into at least two shallow features of the target sample.

[0072] In this embodiment, the preset model can be a Multi-Channel Enhanced Dense Connection Network (MEDCN). After the preset model is trained, a low-resolution image (Low Resolution, LR) can be input into the model to obtain the corresponding high-resolution image (High Resolution, HR); for example... Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of a MEDCN network, including a shallow feature extraction network, a deep feature extraction network, a feature fusion network, an upsampling network, and a convolutional network.

[0073] In this embodiment of the application, given an input low-resolution image LR, a clear high-resolution image HR can be obtained through MEDCN, as shown in the formula:

[0074] I SR =F MEDCN (I LR );

[0075] Where I LR For low-resolution images, F MEDCN (·) represents the function corresponding to MEDCN, I SR For reconstructed high-resolution images.

[0076] In this embodiment, the preset model may include a shallow feature decomposition network. Based on this network, the initial shallow features of the first sample image can be decomposed into at least two target sample shallow features. The dimension of the initial shallow features is greater than the dimension of each target sample shallow feature. The feature vectors corresponding to the initial shallow features can be divided into a preset number of sub-feature vectors according to their dimensions. Each sub-feature vector group has the same number of vectors. A target sample shallow feature is constructed based on each sub-feature vector group. The dimension of each target sample shallow feature is the ratio of the dimension of the initial shallow features to the preset number of vectors. For example, a 64-dimensional initial shallow feature can be decomposed into four groups of 16-dimensional shallow feature vectors, thereby obtaining four second shallow features.

[0077] In this embodiment, the distinction between deep and shallow features primarily depends on the number of convolutional layers. Taking a 3x3 target as an example, the target will disappear after two downsampling iterations; therefore, the target will disappear on Conv3. Conv1 and Conv2 are considered shallow features, while the others are considered deep features. Similarly, for a 5x5 target, conv1, conv2, and conv3 are shallow features, while the others are deep features. Shallow feature extraction consists of a 3×3 convolutional layer containing 64 filters. The low-resolution image is processed through this convolutional layer to obtain a 64-dimensional feature vector. This feature vector is then divided into four equal groups, each used for subsequent deep feature extraction. The corresponding formula is:

[0078] F0 = F S (I LR )

[0079] F 01 ,F 02 ,F 03 ,F 04 =split(F0)

[0080] Where F S(·) represents shallow feature extraction, F0 represents shallow features, and split(·) represents grouping. 01 ,F 02 ,F 03 ,F 04 These are shallow features with a dimension of 16.

[0081] In this embodiment of the application, before decomposing the initial shallow features of the first sample image into at least two shallow features of the target sample based on a preset model, the method further includes:

[0082] Based on the preset model, shallow features of the first sample image are extracted to obtain the initial sample shallow features of the first dimension; the sum of the second dimensions corresponding to the shallow features of the at least two target samples is equal to the first dimension.

[0083] In this embodiment of the application, the first dimension and the second dimension are different. For example, the first dimension can be 64, and the target sample has four shallow features, then the second dimension can be 16.

[0084] In this embodiment of the application, the initial shallow features of the first sample image are decomposed into at least two shallow features of the target sample based on a preset model; the decomposition of features can form a multi-channel network structure, which effectively reduces the parameters of the model while ensuring performance.

[0085] In this embodiment, the preset model may include a shallow feature extraction network. Based on this network, shallow features of the first sample image can be extracted to obtain the initial sample shallow features of the first dimension. The shallow feature extraction network may consist of a first number of convolutional layers, and the first shallow feature may be a feature of the first dimension, such as a 64-dimensional shallow feature.

[0086] S305: Based on the preset model, perform resolution enhancement training on the deep features of the sample to adjust the parameters of the preset model until the resolution loss value of the preset model is less than the preset value; the deep features of the sample are obtained by extracting deep features from the shallow features of the at least two target samples;

[0087] In this embodiment, a resolution loss function corresponding to the preset model can be constructed. Based on the resolution of the image output by the preset model and the resolution of the second sample image, a resolution loss value is determined to train the model. The preset value can be set according to actual conditions. Specifically, the L1 norm can be used as the loss function for model training, and its formula is:

[0088]

[0089] N is the number of training batches. For the i-th original high-resolution image, Let be the i-th reconstructed high-resolution image, and ||·||1 be the 1-norm.

[0090] In the embodiments of this application, such as Figure 3 As shown, the resolution enhancement training of deep features of the sample based on the preset model includes:

[0091] S3051: Based on the preset model, extract the deep features corresponding to the shallow features of each of the at least two target samples to obtain the deep features of at least two samples;

[0092] In this embodiment of the application, the preset model includes a deep feature extraction network. The step of extracting deep features corresponding to the shallow features of each of the at least two target samples based on the preset model to obtain deep features of at least two samples includes:

[0093] Based on the deep feature extraction network, the deep features corresponding to the shallow features of the at least two target samples are extracted to obtain the deep features of the at least two samples.

[0094] In this embodiment, the deep feature extraction network includes at least two enhanced dense connection blocks, which are densely connected to each other. Each enhanced dense connection block is constructed based on a preset number of convolutional layers, and the number of enhanced dense connection blocks is the same as the number of shallow features of the target sample. The method further includes:

[0095] Based on the preset model, the sorting of each of the at least two enhanced dense connection blocks is determined to obtain the connection block sorting result;

[0096] Based on the preset model and the connection block sorting results, the shallow features of the target sample corresponding to each enhanced dense connection block are determined.

[0097] In the embodiments of this application, each enhanced dense connection block corresponds to a shallow feature of a target sample, and each feature enhancement block uses dense connections to effectively avoid the gradient descent problem.

[0098] In this embodiment of the application, the step of extracting the deep features corresponding to the shallow features of the at least two target samples based on the deep feature extraction network to obtain the deep features of at least two samples includes:

[0099] Based on each enhanced dense connection block, deep feature extraction is performed on the shallow features of the target sample corresponding to each enhanced dense connection block to obtain the deep features of the at least two samples.

[0100] In this embodiment, the deep feature extraction network consists of four Enhanced Dense Connection Blocks (EDCBs); wherein the structure of each EDCB is as follows: Figure 8 As shown, EDCB consists of multiple different convolutional networks, each with a different dimension of output channels. 16, 24, 32, ... represent the dimensions of the output channels of the convolutional layer.

[0101] In the embodiments of this application, such as Figure 4 As shown, based on each enhanced dense connection block, deep feature extraction is performed on the shallow features of the target sample corresponding to each enhanced dense connection block to obtain the deep features of the at least two samples, including:

[0102] S401: Input the shallow features of the target sample corresponding to the first enhanced dense connection block into the first enhanced dense connection block to obtain the first deep features; the first enhanced dense connection block is the connection block ranked first among the at least two enhanced dense connection blocks.

[0103] S403: Input the first deep feature and the shallow feature of the target sample corresponding to at least one target augmented dense connection block into the at least one target augmented dense connection block to obtain at least one target deep feature; the target augmented dense connection block is the connection block other than the first augmented dense connection block among the at least two augmented dense connection blocks;

[0104] In this embodiment of the application, the number of enhanced dense connection blocks and the number of shallow features of the target sample are both N, where N≥2 and N is an integer; the step of inputting the first deep feature and the shallow features of the target sample corresponding to at least one target enhanced dense connection block into the at least one target enhanced dense connection block to obtain at least one target deep feature includes:

[0105] The deep features output by each enhanced dense connection block ranked before the Mth enhanced dense connection block, and the shallow features of the target sample corresponding to the Mth enhanced dense connection block, are taken as the Mth feature; the Mth enhanced dense connection block is the connection block ranked Mth among the at least two enhanced dense connection blocks; M = 2, ..., N; M is an integer;

[0106] The Mth feature is input into the Mth enhanced dense connection block to obtain the Mth deep feature;

[0107] The Mth deep feature is taken as the target deep feature.

[0108] In the embodiments of this application, the specific values ​​of M and N can be set according to the actual situation. For example, if N can be 4, then M can be 2, 3, or 4.

[0109] In a specific embodiment, such as Figure 5 As shown, there are four enhanced dense connection blocks. The step of inputting the first deep feature and the shallow feature of the target sample corresponding to at least one target enhanced dense connection block into the at least one target enhanced dense connection block to obtain at least one target deep feature may include:

[0110] S4031: Input the first deep feature and the target sample shallow feature corresponding to the second enhanced dense connection block into the second enhanced dense connection block to obtain the second deep feature; the second enhanced dense connection block is the connection block ranked second among the at least two enhanced dense connection blocks;

[0111] S4033: Input the first deep feature, the second deep feature, and the target sample shallow feature corresponding to the third enhanced dense connection block into the third enhanced dense connection block to obtain the third deep feature; the third enhanced dense connection block is the connection block ranked third among the at least two enhanced dense connection blocks;

[0112] S4035: Input the first deep feature, the second deep feature, the third deep feature, and the target sample shallow feature corresponding to the fourth enhanced dense connection block into the fourth enhanced dense connection block to obtain the fourth deep feature; the fourth enhanced dense connection block is the fourth connection block in the order of the four enhanced dense connection blocks.

[0113] In the embodiments of this application, such as Figure 7 As shown, the i-th EDCB will receive the output information and shallow features of the previous i-1 EDCBs. EDCBs transmit information through dense connections, as expressed by the formula:

[0114]

[0115]

[0116]

[0117]

[0118] in This represents the operation of the i-th EDCB.

[0119] Within each EDCB, the feature information is first convolved through a 16-dimensional output convolutional layer, then subsequent convolutions use dense connections for feature extraction, and finally a 16-dimensional convolutional layer is used for feature fusion.

[0120] S405: The first deep feature and the at least one target deep feature are determined as the sample deep features.

[0121] In this embodiment of the application, when there are four enhanced dense connection blocks, the first deep feature, the second deep feature, the third deep feature, and the fourth deep feature are determined as sample deep features.

[0122] S3053: Based on the preset model, the deep features of the at least two samples are fused to obtain sample fusion features;

[0123] In this embodiment of the application, when there are four enhanced dense connection blocks, the first deep feature, the second deep feature, the third deep feature and the fourth deep feature are fused based on the preset model to obtain sample fusion features.

[0124] In this embodiment, the feature fusion part consists of two 3×3 convolutional layers, and its formula is expressed as follows:

[0125]

[0126]

[0127] Where F u1 (·) represents the first-level fusion operation. For the fused feature information, F u2 (·) represents the second-level fusion operation. This represents the feature information after the second layer of fusion.

[0128] S3055: Perform resolution enhancement training on the sample fusion features based on the preset model.

[0129] In the embodiments of this application, such as Figure 6 As shown, the preset model includes a residual network and a sub-pixel convolutional network. The step of performing resolution enhancement training on the sample fusion features based on the preset model includes:

[0130] S30551: Based on the residual network, perform identity mapping processing on the sample fusion features to obtain sample processing features;

[0131] In this embodiment, ideally, the more layers a neural network has, the easier it is to find this functional relationship. As the number of neural network layers increases, the range of fitable functional relationships increases significantly; however, while the range of fitable functional relationships increases (i.e., the area covered by the last layer), it may become increasingly difficult to fit a function closest to f*. In practical experience, this is reflected in the training results of neural networks as follows: as the number of layers increases, the error rate initially decreases, but then increases. Furthermore, the increased error rate occurs not only on the test set but also on the training set; therefore, this error is unrelated to overfitting.

[0132] Based on traditional neural networks, the output of layer L(L-1) is directly superimposed onto the output of layer L. That is:

[0133] f(x)=L(x)+x

[0134] At this point, the range of the function that the L-layer neural network can fit must be the union of the range of the function that the L-th layer neural network can fit and the range of the function that the (L-1)-th layer neural network can fit. The identity mapping refers to the fact that when L(x)→0, f(x)=x; at this time, the range of the function that the L-layer neural network can fit must completely cover the range of the function that the (L-1)-th layer neural network can fit; and a neural network with this structure is a residual network.

[0135] In this embodiment, the sample fusion features can be processed by an identity mapping using a global residual network to obtain the sample processing features; the corresponding formula is:

[0136]

[0137] Where F u These are the features after processing by the residual network.

[0138] S30553: Based on the sub-pixel convolutional network, the sample processing features are upsampled for training; the upsampling is used to enhance the resolution of the sample processing features.

[0139] In this embodiment, during camera imaging, the acquired image data is discretized. Due to the limitations of the photosensitive element itself, each pixel on the imaging surface only represents the nearby color. For example, there may be a 4.5µm gap between pixels on two photosensitive elements. Macroscopically, they are connected, but microscopically, countless tiny particles exist between them. These pixels existing between two actual physical pixels are called "subpixels." Subpixels actually exist, but they lack a smaller sensor to detect them, so they can only be approximated in software. The precision of subpixels can be adjusted according to the interpolation between adjacent pixels. For example, a quarter-pixel means treating each pixel as four pixels horizontally and vertically. In this way, subpixel interpolation can achieve a mapping from small rectangles to large rectangles, thereby improving resolution.

[0140] Upsampling is the process of acquiring samples of an analog signal. Sampling transforms a signal that is continuous in both time and amplitude into a discrete signal in both time and amplitude under the action of a sampling pulse. Therefore, sampling is also known as the discretization process of a waveform.

[0141] Upsampling is performed through subpixel convolution to obtain a reconstructed high-resolution image, expressed by the formula:

[0142] I SR =F upsample (F u )

[0143] Where F upsample (·) represents a subpixel convolution operation, I SR To reconstruct high-resolution images.

[0144] S307: Use the preset model corresponding to the current model parameters as the image processing model; the current model parameters are the model parameters when the resolution loss value is less than the preset value.

[0145] In this embodiment, during the training of the image processing model, validation data can also be used to validate the model; after each epoch, the accuracy of the current model is tested. One epoch represents the completion of one forward computation and backpropagation process by feeding all the training data into the network.

[0146] In this embodiment of the application, after the image processing model has been trained, test data can be used to test the accuracy of the model; if the accuracy of the model is low, it needs to be retrained to ensure the accuracy of the model.

[0147] In a specific embodiment, such as Figure 9 As shown, Figure 9The area within the rectangular box is a schematic diagram of the image to be processed; this image is a low-resolution image. Figure 10 These are schematic diagrams illustrating high-resolution images obtained using different methods. Image processing models were constructed using Bicubic (bicubic interpolation network), SRCNN (Super-Resolution Convolutional Neural Network), VDSR (Accurate Image Super-Resolution Using Very Deep Convolutional Networks), LapSRN (Laplace Static Random Network), and MEDCN in this embodiment, respectively. These models were used to process... Figure 9 By performing resolution enhancement processing and enlarging the processed image four times, the desired result can be obtained. Figure 10 The images shown have resolutions of 21.98dB, 24.91dB, 25.70dB, 25.79dB, and 26.44dB respectively. It can be seen that the MEDCN model trained using the method of this embodiment produces the highest resolution and the clearest images.

[0148] In a specific embodiment, after enlarging the images in the Set5 test set (super-resolution dataset) by three times, the parameters of the image processing models trained using different networks are shown in Table 1 below. These models include SRCNN, VDSR, MEDCN, FSRCNN (Fast Super-Resolution Convolutional Neural Network), DRCN (Deeply-Recursive Convolutional Network), DRRN (Deep Recursive Residual Network), MemNet (Deep Persistent Memory Network), and IDN (Information Distillation Network). It can be seen that the MEDCN model in this embodiment does not have the maximum memory and depth values, but it has the highest PSNR and a relatively high SSIM. Model depth represents the number of convolutional layers in the model. Compared with other deep learning-based image super-resolution network models, the MEDCN model in this embodiment achieves higher resolution and clarity of the processed images with smaller model parameters and computational load. This embodiment can achieve better reconstruction performance with fewer parameters, and has certain improvements in objective evaluation indicators PSNR, SSIM and subjective visual effects.

[0149] Table 1

[0150] method Model memory Model depth PSNR SSIM SRCNN 8K 3 32.75 0.9090 FSRCNN 13K 8 33.18 0.9140 VDSR 666K 20 33.66 0.9213 DRCN 1774K 20 33.82 0.9226 DRRN 298K 52 34.03 0.9244 MemNet 678K 80 34.09 0.9248 IDN 553K 31 34.11 0.9253 MEDCN 665K 34 34.14 0.9243

[0151] The image processing method of this application can be applied to many different fields such as medical diagnosis, satellite imaging, and road monitoring.

[0152] To address the issue of large parameter counts and high computational resource and time consumption during network training in existing neural network models, this application effectively solves the problem of large model parameter count by grouping shallow features into multi-channel networks, reducing the number of subsequent feature maps. Simultaneously, the lightweight enhanced dense connection block structure in this application is used to extract deep network features, effectively preventing the loss of shallow information during transmission, solving the gradient vanishing problem during backpropagation, and expanding the receptive field of convolutional layers. In convolutional neural networks, the receptive field is defined as the area mapped to the input image by a pixel in the feature map output by each layer of the convolutional neural network. That is, a point in the feature map corresponds to a region in the input image. Furthermore, dense connections are used between feature enhancement blocks to strengthen the connection between shallow and deep features, reducing training difficulty.

[0153] As can be seen from the technical solutions provided by the embodiments of this application above, the embodiments of this application acquire an image to be processed with a resolution less than a preset threshold; perform resolution enhancement processing on the image to be processed based on an image processing model to obtain a target image; wherein, the training method of the image processing model includes: acquiring a first sample image and a second sample image; decomposing the initial sample shallow features of the first sample image into at least two target sample shallow features based on a preset model; performing resolution enhancement training on the sample deep features based on the preset model to adjust the parameters of the preset model until the resolution loss value of the preset model is less than the preset value; the sample deep features are obtained by extracting deep features from the at least two target sample shallow features; using the preset model corresponding to the current model parameters as the image processing model; the current model parameters are the model parameters when the resolution loss value is less than the preset value. In the training process of the model, this application decomposes the shallow features extracted from the image and then extracts deep features based on the decomposed shallow features for model training. This reduces the model parameters and the amount of computation during model training while ensuring the model accuracy, thereby improving the model training speed. Furthermore, the resulting image processing model can accurately identify the detailed information in the image to be processed, thus improving the clarity of the target image.

[0154] This application also provides an image processing apparatus, such as... Figure 11 As shown, the device includes:

[0155] The image acquisition module 1110 is used to acquire images to be processed with a resolution less than a preset threshold.

[0156] Image processing module 1120 is used to perform resolution enhancement processing on the image to be processed based on an image processing model to obtain a target image; the resolution of the target image is greater than the resolution of the image to be processed.

[0157] The sample image acquisition module 1130 is used to acquire a first sample image and a second sample image; the first sample image is obtained by reducing the resolution of the second sample image; the resolution of the second sample image is greater than the preset threshold.

[0158] The sample shallow feature decomposition module 1140 is used to decompose the initial sample shallow features of the first sample image into at least two target sample shallow features based on a preset model.

[0159] The training module 1150 is used to perform resolution enhancement training on the deep features of the samples based on the preset model, so as to adjust the parameters of the preset model until the resolution loss value of the preset model is less than the preset value; the deep features of the samples are obtained by extracting deep features from the shallow features of the at least two target samples;

[0160] The model determination module 1160 is used to take the preset model corresponding to the current model parameters as the image processing model; the current model parameters are the model parameters when the resolution loss value is less than the preset value.

[0161] In some embodiments, the apparatus may further include:

[0162] The initial sample shallow feature extraction module is used to extract shallow features of the first sample image based on the preset model to obtain the initial sample shallow features of the first dimension; the sum of the second dimensions corresponding to the shallow features of the at least two target samples is equal to the first dimension.

[0163] In some embodiments, the training module may include:

[0164] The sample deep feature determination submodule is used to extract the deep features corresponding to the shallow features of the at least two target samples based on the preset model, so as to obtain at least two sample deep features;

[0165] The sample fusion feature determination submodule is used to fuse the deep features of the at least two samples based on the preset model to obtain sample fusion features;

[0166] The training submodule is used to perform resolution enhancement training on the sample fusion features based on the preset model.

[0167] In some embodiments, the preset model includes a deep feature extraction network, and the sample deep feature determination submodule may include:

[0168] The deep feature extraction submodule is used to extract the deep features corresponding to the shallow features of the at least two target samples based on the deep feature extraction network, so as to obtain the deep features of at least two samples.

[0169] In some embodiments, the deep feature extraction network includes at least two enhanced dense connection blocks, the number of which is the same as the number of shallow features of the target sample; the apparatus may further include:

[0170] The connection block sorting result determination module is used to determine the sorting of each of the at least two enhanced dense connection blocks based on the preset model, and to obtain the connection block sorting result.

[0171] The target sample shallow feature determination module is used to determine the target sample shallow features corresponding to each enhanced dense connection block based on the preset model and the connection block sorting results.

[0172] In some embodiments, the deep feature extraction submodule may include:

[0173] The deep feature extraction unit is used to extract deep features from the shallow features of the target sample corresponding to each enhanced dense connection block based on each enhanced dense connection block, so as to obtain the deep features of the at least two samples.

[0174] In some embodiments, the deep feature extraction unit may include:

[0175] The first deep feature determination subunit is used to input the shallow features of the target sample corresponding to the first enhanced dense connection block into the first enhanced dense connection block to obtain the first deep feature; the first enhanced dense connection block is the connection block ranked first among the at least two enhanced dense connection blocks.

[0176] The target deep feature determination subunit is used to input the first deep feature and the target sample shallow feature corresponding to at least one target enhanced dense connection block into the at least one target enhanced dense connection block to obtain at least one target deep feature; the target enhanced dense connection block is the connection block other than the first enhanced dense connection block among the at least two enhanced dense connection blocks.

[0177] The sample deep feature determination subunit is used to determine the first deep feature and the at least one target deep feature as the sample deep feature.

[0178] In some embodiments, the number of enhanced dense connection blocks and the number of shallow features of the target sample are both N, where N≥2 and N is an integer; the target deep feature determination subunit may include:

[0179] The Mth feature determination subunit is used to take the deep features output by each enhanced dense connection block that is ranked before the Mth enhanced dense connection block, and the shallow features of the target sample corresponding to the Mth enhanced dense connection block, as the Mth feature; the Mth enhanced dense connection block is the connection block ranked Mth among the at least two enhanced dense connection blocks; where M = 2, ..., N, and M is an integer;

[0180] The Mth deep feature determination subunit is used to input the Mth feature into the Mth enhanced dense connection block to obtain the Mth deep feature;

[0181] A feature determination subunit is used to take the Mth deep feature as the target deep feature.

[0182] In some embodiments, the preset model includes a residual network and a subpixel convolutional network, and the apparatus may further include:

[0183] In some embodiments, the training submodule may include:

[0184] The sample processing feature determination unit is used to perform identity mapping processing on the sample fusion features based on the residual network to obtain sample processing features;

[0185] An upsampling training unit is used to upsample the sample processing features based on the subpixel convolutional network; the upsampling is used to enhance the resolution of the sample processing features.

[0186] The apparatus and method embodiments described herein are based on the same inventive concept.

[0187] This application provides an image processing device, which includes a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement the image processing method provided in the above method embodiments.

[0188] Embodiments of this application also provide a computer storage medium, which can be disposed in a terminal to store at least one instruction or at least one program related to implementing an image processing method in the method embodiments. The at least one instruction or at least one program is loaded and executed by the processor to implement the image processing method provided in the above method embodiments.

[0189] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing method provided in the above-described method embodiments.

[0190] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0191] The memory described in this application embodiment can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for the functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0192] The image processing method embodiments provided in this application can be executed on a mobile terminal, computer terminal, server, or similar computing device. Taking running on a server as an example, Figure 12 This is a hardware structure block diagram of a server for an image processing method provided in an embodiment of this application. For example... Figure 12As shown, the server 1200 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 1210 (CPUs 1210 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 1230 for storing data, and one or more storage media 1220 (e.g., one or more mass storage devices) for storing application programs 1223 or data 1222. The memory 1230 and storage media 1220 may be temporary or persistent storage. The program stored in the storage media 1220 may include one or more modules, each module including a series of instruction operations on the server. Furthermore, the CPU 1210 may be configured to communicate with the storage media 1220 and execute the series of instruction operations stored in the storage media 1220 on the server 1200. Server 1200 may also include one or more power supplies 1260, one or more wired or wireless network interfaces 1250, one or more input / output interfaces 1240, and / or one or more operating systems 1221, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0193] The input / output interface 1240 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 1200. In one example, the input / output interface 1240 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 1240 may be a radio frequency (RF) module for wireless communication with the Internet.

[0194] Those skilled in the art will understand that Figure 12 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 1200 may also include... Figure 12 The more or fewer components shown, or having the same Figure 12 The different configurations shown.

[0195] As can be seen from the embodiments of the image processing method, apparatus, device, or storage medium provided in this application above, this application acquires an image to be processed with a resolution less than a preset threshold; performs resolution enhancement processing on the image to be processed based on an image processing model to obtain a target image; wherein, the training method of the image processing model includes: acquiring a first sample image and a second sample image; decomposing the initial sample shallow features of the first sample image into at least two target sample shallow features based on a preset model; performing resolution enhancement training on the sample deep features based on the preset model to adjust the parameters of the preset model until the resolution loss value of the preset model is less than the preset value; the sample deep features are obtained by extracting deep features from the at least two target sample shallow features; and using the preset model corresponding to the current model parameters as the image processing model; the current model parameters are the model parameters when the resolution loss value is less than the preset value. In the training process of the model, this application decomposes the shallow features extracted from the image and then extracts deep features based on the decomposed shallow features for model training. This reduces the model parameters and the amount of computation during model training while ensuring the model accuracy, thereby improving the model training speed. Furthermore, the resulting image processing model can accurately identify the detailed information in the image to be processed, thus improving the clarity of the target image.

[0196] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0197] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0198] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer storage medium, such as a read-only memory, a disk, or an optical disk.

[0199] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a to-be-processed image with a resolution less than a preset threshold; performing resolution enhancement processing on the to-be-processed image based on an image processing model to obtain a target image; the resolution of the target image is greater than that of the to-be-processed image; wherein the training method of the image processing model comprises: acquiring a first sample image and a second sample image; the first sample image is obtained based on resolution reduction processing on the second sample image; decomposing initial sample shallow features of the first sample image into at least two target sample shallow features based on a preset model; the dimensions of each target sample shallow feature are the same; the preset model comprises at least two enhanced dense connection blocks; the number of the enhanced dense connection blocks and the number of the target sample shallow features are both N, N≥2 and N is an integer; each enhanced dense connection block corresponds to a target sample shallow feature; inputting the target sample shallow feature corresponding to a first enhanced dense connection block into the first enhanced dense connection block to obtain a first deep feature; the first enhanced dense connection block is the first connection block in the order among the at least two enhanced dense connection blocks; outputting deep features output by each enhanced dense connection block located before an Mth enhanced dense connection block and the target sample shallow feature corresponding to the Mth enhanced dense connection block as an Mth feature; the Mth enhanced dense connection block is the Mth connection block in the order among the at least two enhanced dense connection blocks; wherein M=2, …, N, and M is an integer; inputting the Mth feature into the Mth enhanced dense connection block to obtain an Mth deep feature; taking the Mth deep feature as a target deep feature, and taking the first deep feature and at least one target deep feature as sample deep features; performing resolution enhancement training on the sample deep features based on the preset model to adjust parameters of the preset model until a resolution loss value of the preset model is less than a preset value; taking a preset model corresponding to current model parameters as the image processing model; the current model parameters are model parameters when the resolution loss value is less than the preset value.

2. The method of claim 1, wherein, Before the preset model is used to decompose the initial sample shallow features of the first sample image into at least two target sample shallow features, the method further comprises: extracting shallow features of the first sample image based on the preset model to obtain the initial sample shallow features with a first dimension; the sum of second dimensions corresponding to the at least two target sample shallow features is equal to the first dimension.

3. The method of claim 1, wherein, The resolution enhancement training on the sample deep features based on the preset model comprises: extracting deep features corresponding to the at least two target sample shallow features based on the preset model to obtain at least two sample deep features; performing fusion processing on the at least two sample deep features based on the preset model to obtain a sample fusion feature; performing resolution enhancement training on the sample fusion feature based on the preset model.

4. The method of claim 3, wherein, The preset model comprises a deep feature extraction network, and the deep features corresponding to the at least two target sample shallow features are extracted based on the preset model to obtain at least two sample deep features. The deep features corresponding to the at least two target sample shallow features are extracted based on the deep feature extraction network to obtain at least two sample deep features.

5. The method of claim 4, wherein, The method further comprises: The ordering of the at least two enhanced dense connection blocks is determined based on the preset model to obtain a connection block ordering result. The target sample shallow feature corresponding to each enhanced dense connection block is determined based on the preset model and the connection block ordering result. The deep features corresponding to the at least two target sample shallow features are extracted based on the deep feature extraction network to obtain at least two sample deep features, comprising: The deep features corresponding to the target sample shallow feature corresponding to each enhanced dense connection block are extracted based on each enhanced dense connection block to obtain the at least two sample deep features.

6. The method of claim 3, wherein, The preset model comprises a residual network and a sub-pixel convolution network, and the sample fusion feature is subjected to resolution enhancement training based on the preset model, comprising: The sample fusion feature is subjected to identity mapping processing based on the residual network to obtain a sample processing feature. The sample processing feature is subjected to up-sampling training based on the sub-pixel convolution network; the up-sampling is used to enhance the resolution of the sample processing feature.

7. An image processing apparatus characterized by comprising: The device comprises: A to-be-processed image acquisition module configured to acquire a to-be-processed image with a resolution less than a preset threshold; An image processing module configured to perform resolution enhancement processing on the to-be-processed image based on an image processing model to obtain a target image; the resolution of the target image is greater than the resolution of the to-be-processed image; A sample image acquisition module configured to acquire a first sample image and a second sample image; the first sample image is obtained based on resolution reduction processing on the second sample image; the resolution of the second sample image is greater than the preset threshold; A sample shallow feature decomposition module configured to decompose an initial sample shallow feature of the first sample image into at least two target sample shallow features based on a preset model; the dimensions of each target sample shallow feature are the same; the preset model comprises at least two enhanced dense connection blocks; the number of the enhanced dense connection blocks and the number of the target sample shallow features are both N, N≥2 and N is an integer; each enhanced dense connection block corresponds to a target sample shallow feature; The training module is configured to input target sample shallow features corresponding to a first enhanced dense connection block into the first enhanced dense connection block to obtain a first deep feature; the first enhanced dense connection block is a connection block ranked first among the at least two enhanced dense connection blocks; deep features output by each enhanced dense connection block ranked before an Mth enhanced dense connection block and target sample shallow features corresponding to the Mth enhanced dense connection block are taken as Mth features; the Mth enhanced dense connection block is a connection block ranked Mth among the at least two enhanced dense connection blocks; M is an integer and M=2, …, N; the Mth features are input into the Mth enhanced dense connection block to obtain Mth deep features; the Mth deep features are taken as target deep features, and the first deep features and at least one target deep feature are determined as sample deep features; and the sample deep features are subjected to resolution enhancement training based on the preset model to adjust parameters of the preset model until a resolution loss value of the preset model is less than a preset value. The model determination module is configured to take a preset model corresponding to current model parameters as the image processing model; the current model parameters are model parameters when the resolution loss value is less than the preset value.

8. The apparatus of claim 7, wherein, The device further includes: The initial sample shallow feature extraction module is configured to extract shallow features of the first sample image based on the preset model to obtain initial sample shallow features of a first dimension; and a sum of second dimensions corresponding to the at least two target sample shallow features is equal to the first dimension.

9. The apparatus of claim 7, wherein, The training module includes: The sample deep feature determination submodule is configured to extract deep features corresponding to the at least two target sample shallow features based on the preset model to obtain at least two sample deep features. The sample fusion feature determination submodule is configured to perform fusion processing on the at least two sample deep features based on the preset model to obtain a sample fusion feature. The training submodule is configured to perform resolution enhancement training on the sample fusion feature based on the preset model.

10. The apparatus of claim 9, wherein, The preset model includes a deep feature extraction network, and the sample deep feature determination submodule includes: The deep feature extraction submodule is configured to extract deep features corresponding to the at least two target sample shallow features based on the deep feature extraction network to obtain at least two sample deep features.

11. The apparatus of claim 10, wherein, The device further includes: The connection block ranking result determination module is configured to determine rankings corresponding to the at least two enhanced dense connection blocks based on the preset model to obtain a connection block ranking result. The target sample shallow feature determination module is configured to determine target sample shallow features corresponding to each enhanced dense connection block based on the preset model and the connection block ranking result.

12. The apparatus of claim 10, wherein, The deep feature extraction submodule includes: The deep feature extraction unit is configured to perform deep feature extraction on the target sample shallow features corresponding to each enhanced dense connection block based on each enhanced dense connection block to obtain the at least two sample deep features.

13. The apparatus of claim 9, wherein, The preset model comprises a residual network and a sub-pixel convolution network, and the training submodule comprises: A sample processing feature determination unit is configured to perform identity mapping processing on the sample fusion feature based on the residual network to obtain a sample processing feature. An up-sampling training unit is configured to perform up-sampling training on the sample processing feature based on the sub-pixel convolution network. The up-sampling is configured to enhance the resolution of the sample processing feature.

14. An image processing apparatus characterized by comprising: The device comprises a processor and a memory, and the memory stores at least one instruction or at least one program. The at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method according to any one of claims 1-6.

15. A computer storage medium, comprising, The computer storage medium stores at least one instruction or at least one program. The at least one instruction or the at least one program is loaded and executed by the processor to implement the image processing method according to any one of claims 1-6.

16. A computer program product comprising computer instructions, characterized in that, The computer instruction is executed by the processor to implement the image processing method according to any one of claims 1-6.

Images