Network training method and apparatus, storage medium, and computer program
By extracting multi-scale features and removing redundant information, the network is trained to generate high-resolution images, solving the problem of low image resolution in oil extraction and improving the accuracy of image analysis.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA PETROLEUM & CHEMICAL CORP
- Filing Date
- 2024-11-27
- Publication Date
- 2026-05-29
AI Technical Summary
During oil extraction, the low resolution of geological images captured by airborne equipment leads to insufficient accuracy in image analysis.
By acquiring low-resolution and high-resolution images, multi-scale features are extracted, redundant information is removed, images are reconstructed, and a network is trained based on the difference to generate high-resolution images.
This improves the accuracy of generating high-resolution images, thereby improving the accuracy of analysis based on high-resolution images.
Smart Images
Figure CN122115204A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of oil and gas extraction technology, and in particular to a network training method, apparatus, storage medium, and computer program. Background Technology
[0002] During oil extraction, images of the geological formations are captured using airborne equipment and space-based acquisition instruments. However, due to the poor optical conditions of the formations, the resolution of the captured images is low. Summary of the Invention
[0003] This disclosure provides a network training method, apparatus, storage medium, and computer program. After acquiring a first image and a second image, a third image can be reconstructed by extracting multi-scale features and deleting redundant information. The network is trained based on the differences between the first image and the third image, ensuring that the trained network can generate high-resolution images from low-resolution images, thereby improving the accuracy of generating high-resolution images and thus improving the accuracy of analysis based on high-resolution images.
[0004] Firstly, this disclosure provides a network training method, including:
[0005] Obtain a training dataset, which includes a first image and a second image, wherein the second image is obtained based on the first image and the resolution of the second image is lower than that of the first image;
[0006] The second image is input into the network, and features are extracted from the second image to obtain multi-scale image features, wherein the multi-scale image features are generated based on image features at different scales;
[0007] Redundant information in the multi-scale image features is deleted and reconstructed to obtain a third image corresponding to the second image;
[0008] The network is trained based on the differences between the first image and the third image.
[0009] In some embodiments, inputting the second image into the network and extracting features from the second image to obtain multi-scale image features includes:
[0010] The second image is input into the network, and features are extracted from the second image based on multiple scales to obtain multiple types of image features, wherein each type of image feature corresponds to a scale;
[0011] The multi-scale image features are obtained by concatenating the various image features and integrating them through convolution.
[0012] In some embodiments, the step of inputting the second image into the network and extracting features from the second image based on multiple scales to obtain multiple types of image features includes:
[0013] The second image is input into the network, and the second image is reduced to obtain the reduction feature;
[0014] The second image is subjected to adaptive averaging, convolution, and weight assignment to obtain contextual features.
[0015] The image features are obtained by convolution and fusion of the reduced features and the context features.
[0016] In some embodiments, the step of deleting and reconstructing redundant information in the multi-scale image features to obtain a third image corresponding to the second image includes:
[0017] The redundant information of the multi-scale image features is removed based on the octave convolution structure, and the third image is reconstructed.
[0018] In some embodiments, each degree of the octave convolutional structure includes:
[0019] The multi-scale image features are decomposed to obtain high-frequency features and low-frequency features;
[0020] The high-frequency features are convolved and the low-frequency features are interpolated to obtain the high-frequency updated features;
[0021] The low-frequency features are convolved and the high-frequency features are averaged pooled to obtain the low-frequency updated features.
[0022] In some embodiments, the method further includes:
[0023] The first image is blurred to obtain a blurred image;
[0024] The blurred image is scaled to obtain a scaled image;
[0025] The scaled image is denoised to obtain a denoised image;
[0026] The denoised image is compressed to obtain the second image.
[0027] Secondly, this disclosure provides a network training device, comprising:
[0028] The data acquisition module is used to acquire a training dataset, which includes a first image and a second image, wherein the second image is acquired based on the first image and the resolution of the second image is lower than that of the first image;
[0029] The feature extraction module is used to input the second image into the network, extract features from the second image, and obtain multi-scale image features, wherein the multi-scale image features are generated based on image features of different scales;
[0030] The processing module is used to delete and reconstruct redundant information in the multi-scale image features to obtain a third image corresponding to the second image;
[0031] A training module is used to train the network based on the differences between the first image and the third image.
[0032] Thirdly, this disclosure provides a computer device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the foregoing aspects.
[0033] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the methods described in the above aspects.
[0034] Fifthly, this disclosure provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods described in the foregoing aspects.
[0035] This disclosure provides a network training method, apparatus, device, storage medium, and computer program. After acquiring a first image and a second image, a third image can be reconstructed by extracting multi-scale features and deleting redundant information. The network is trained based on the differences between the first image and the third image, ensuring that the trained network can generate high-resolution images from low-resolution images, improving the accuracy of generating high-resolution images, and thus improving the accuracy of analysis based on high-resolution images.
[0036] 1. Technical Features: The second image is input into the network, and features are extracted from the second image based on multiple scales to obtain multiple types of image features, wherein each type of image feature corresponds to a scale; the multiple types of image features are concatenated and integrated through convolution to obtain the multi-scale image features. The technical effect is that image features at multiple scales can be obtained, expanding the diversity of the obtained scales, thereby improving the accuracy of subsequent network training.
[0037] 2. Technical Features: The second image is input into the network, and the second image is reduced to obtain reduced features; adaptive averaging, convolution, and weight assignment are performed on the second image to obtain context features; convolution and fusion are performed based on the reduced features and the context features to obtain the image features. The technical effect is to solve the problem of feature extraction according to the scale corresponding to the weights, improve the accuracy of the extracted image features, and thus improve the accuracy of subsequent network training.
[0038] 3. The technical feature calls the network to delete redundant information of the multi-scale image features based on the octave convolution structure and reconstruct the third image. The technical effect is to solve the problem that too much redundant information affects network training and improve the accuracy of subsequent network training.
[0039] 4. Technical Features: The multi-scale image features are decomposed into high-frequency and low-frequency features; the high-frequency features are convolved and the low-frequency features are interpolated to obtain high-frequency update features; the low-frequency features are convolved and the high-frequency features are average pooled to obtain low-frequency update features. The technical effect is to reduce redundant information interference through high and low-frequency features, improve the accuracy of the acquired image features, and thus improve the accuracy of subsequent network training.
[0040] 5. Technical Features: The first image is blurred to obtain a blurred image; the blurred image is scaled to obtain a scaled image; the scaled image is denoised to obtain a denoised image; the denoised image is compressed to obtain a second image. The technical effect is that by performing multiple operations on the first image to achieve degradation, the reliability of the data used to train the network is improved, thereby improving the accuracy of training the network. Attached Figure Description
[0041] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:
[0042] Figure 1 A schematic flowchart of a network training method provided in an embodiment of this disclosure;
[0043] Figure 2 A schematic diagram illustrating the principle of a DCM provided in this embodiment of the disclosure;
[0044] Figure 3 A schematic diagram illustrating the updating of high-frequency and low-frequency features according to an embodiment of this disclosure;
[0045] Figure 4 This is a flowchart illustrating a network training method provided in an embodiment of the present disclosure.
[0046] In the accompanying drawings, the same parts are referred to by the same reference numerals, and the drawings are not drawn to scale. Detailed Implementation
[0047] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.
[0048] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0049] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0050] Figure 1 This is a flowchart illustrating a network training method provided in an embodiment of this disclosure. Figure 1 As shown, a network training method includes:
[0051] Step S101: The computer device acquires the training dataset.
[0052] In some embodiments, the training dataset is used to train the network. Alternatively, it can be understood as training a network with the ability to improve image resolution using the training dataset. In some embodiments, the network is used to transform low-resolution images into high-resolution images. Alternatively, it can be understood as the network being used to improve the resolution of images.
[0053] In some embodiments, the name of the network is not limited, and it may be, for example, an image processing network, a resolution enhancement network, etc.
[0054] In some embodiments, the training dataset includes a first image and a second image. Optionally, the first image has a higher resolution than the second image. Alternatively, the first image can be understood as a high-resolution image. Optionally, the second image is obtained based on the first image. Alternatively, the second image can be obtained by degrading the first image. Degradation can also be understood as reducing the resolution of the image.
[0055] In some embodiments, the database stores multiple first images, and the computer device can retrieve the first images from the database. Optionally, the database refers to the database of the computer device itself, the database of the server associated with the computer device, or a database set up in other ways; this disclosure does not limit the scope of the embodiments.
[0056] In some embodiments, the computer device acquires the second image by: blurring the first image to obtain a blurred image, scaling the blurred image to obtain a scaled image, denoising the scaled image to obtain a denoised image, and compressing the denoised image to obtain the second image.
[0057] It should be noted that, in the embodiments of this disclosure, the computer device can also perform ringing and overtuning artifact operations on the denoised image based on the filter to obtain a second image.
[0058] Optionally, the computer device inputs the first image into the degradation model and performs degradation processing on the first image using the above method to obtain the second image.
[0059] In this embodiment of the disclosure, after acquiring the first image and the second image, the computer device can train the network based on the first image and the second image.
[0060] It should be noted that the first image in this embodiment includes images captured during the petroleum and petrochemical production process. Alternatively, it can be understood that the network trained in this embodiment can be applied to the petroleum and petrochemical production process, and the resolution of images during the process can be enhanced through this network, thereby effectively improving the spatial resolution of the images and restoring texture details, thus improving the production efficiency of the petroleum and petrochemical process.
[0061] In step S102, the computer device inputs the second image into the network, extracts features from the second image, and obtains multi-scale image features.
[0062] In some embodiments, multi-scale image features are generated based on image features at different scales. Optionally, image features at different scales refer to features obtained by extracting features from the second image using different convolution kernels.
[0063] In some embodiments, the name of the multi-scale image feature is not limited. It may be, for example, a feature map, an image feature, a multi-scale feature, etc.
[0064] Optionally, the network includes ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) and DCM (Discrete Choise Model), through which multi-scale image features are extracted.
[0065] In this embodiment of the disclosure, a computer device can perform feature extraction on the second image based on the network to obtain multi-scale image features.
[0066] In step S103, the computer device deletes and reconstructs redundant information in the multi-scale image features to obtain the third image corresponding to the second image.
[0067] In this embodiment of the disclosure, since the computer device acquires the second image through the first image, redundant information is generated when extracting multi-scale image features from the second image. The redundant information will affect the accuracy of subsequent training of the network. Therefore, the redundant information in the multi-scale image features is deleted and reconstructed to obtain the corresponding third image, which can improve the accuracy of the obtained third image.
[0068] In step S104, the computer device trains the network based on the differences between the first image and the third image.
[0069] In this embodiment of the present disclosure, after the network processes the low-resolution image through the above steps S102-S103, there is still a difference between the third image and the first image. The computer device in this embodiment of the present disclosure can adjust the network by obtaining the difference between the first image and the third image, so as to enable the network to enhance the second image into the third image.
[0070] In step S105, the computer device enhances the fourth image based on the trained network to obtain the fifth image corresponding to the fourth image.
[0071] In some embodiments, the resolution of the fifth image is higher than that of the fourth image.
[0072] In this embodiment of the disclosure, after the network is trained, the network has the ability to enhance low-resolution images. Therefore, after the computer device inputs the fourth image into the network, it can process the fourth image based on the network to obtain the fifth image.
[0073] It should be noted that step S105 in this embodiment is an example, and in another embodiment, step S105 may not be performed.
[0074] The method provided in this disclosure, after acquiring a first image and a second image, can reconstruct a third image by extracting multi-scale features and deleting redundant information. The network is trained based on the difference between the first image and the high-resolution image to ensure that the trained network can generate the third image, thereby improving the accuracy of generating the third image and thus improving the accuracy of analysis based on the third image.
[0075] Based on the above embodiment, step S102 can be replaced by: the computer device inputs the second image into the network, extracts features from the second image based on multiple scales to obtain multiple types of image features, and concatenates the multiple types of image features and integrates them through convolution to obtain multi-scale image features. Each type of image feature corresponds to one scale.
[0076] Optionally, the computer device extracts multi-scale image features using a DCM (Digital Machine Module). For example, each DCM can extract image features at one scale, and multiple DCMs can extract image features at multiple scales. It should be noted that the second image can also be considered as an image feature at one scale.
[0077] Optionally, the network is invoked to extract features from the second image at multiple scales to obtain multiple types of image features, including: invoking the network to reduce the second image to obtain reduced features; invoking the network to perform adaptive averaging, convolution, and weight assignment on the second image to obtain context features; and convolution and fusing the reduced features and context features to obtain image features.
[0078] Alternatively, assume the input feature map is R. X ∈R c×h×w Where h, w, and c are the height, width, and number of channels, respectively, and three DCM modules are used to extract features from Rx in parallel to obtain D1∈R. c×h×w D2∈R c×h×w ,D3∈R c×h×w Here, DCM is a context-aware filter with convolutional kernel sizes of 1×1, 3×3, and 5×5. For R... X D1, D2, and D3 are concatenated, and the concatenated features are further integrated through a convolutional layer, thereby effectively fusing the captured dynamic multi-scale contextual information.
[0079] Optionally, the principles of DCM are explained below, for example, see [link to relevant documentation]. Figure 2 The diagram first uses convolutional layers to simplify the input, resulting in a reduced feature map.
[0080] in,
[0081] Then, contextual features are extracted through adaptive average pooling and 1×1 convolution, and corresponding weights are assigned to the context-aware filter gk(x) to enable the network structure to adapt to the input image.
[0082]
[0083] The obtained context-aware filter and the reduced feature map R are then compared. reduce Perform convolution to obtain
[0084]
[0085]
[0086] in, This represents matrix multiplication.
[0087] Finally, a 1×1 convolution is applied to fuse the channel information, resulting in a specific output.
[0088]
[0089] This embodiment of the disclosure solves the problem of feature extraction according to the scale corresponding to the weights, which improves the accuracy of the extracted image features and thus improves the accuracy of subsequent network training.
[0090] Based on the above embodiment, step S103 can be replaced by: calling the network, deleting redundant information of multi-scale image features based on the octave convolution structure and reconstructing the third image.
[0091] In some embodiments, each degree structure in the octave convolutional structure includes: decomposing multi-scale image features to obtain high-frequency features and low-frequency features, convolving the high-frequency features and interpolating the low-frequency features to obtain high-frequency update features, and convolving the low-frequency features and performing average pooling on the high-frequency features to obtain low-frequency update features.
[0092] Optionally, the image features in the embodiments of this disclosure can also be understood as having a corresponding frequency domain map, which can be divided into a low-frequency part and a high-frequency part, thereby obtaining a new multi-frequency feature representation method. Specifically, high-frequency image features and low-frequency image features can be mapped and stored in different groups.
[0093] For example, X = {X H ,X L}, where X∈R c×h×w Let X represent the input feature tensor. H ∈R (1-α)×c×h×w X represents the high-frequency feature map after decomposition. L ∈R α×c×h×w The diagram represents the decomposed low-frequency feature map, where h, w, and c are the height, width, and number of channels, respectively, and α represents the ratio of channels allocated to the low-frequency portion. For example, α can be 0.5, 0.4, or other values, and this disclosure does not limit this.
[0094] In some embodiments, the receptive field of octave convolution increases because the resolution of low-frequency image features is reduced. When processing low-frequency feature maps using convolution kernels, the receptive field of octave convolution is almost twice that of ordinary convolution, which can further help octave convolutional layers capture long-distance contextual information. At the same time, the communication between high and low frequencies in octave convolution promotes the utilization of multi-scale image features, thereby possessing multi-scale feature fusion capabilities at the convolutional level.
[0095] Optionally, see Figure 3 As shown, the high-frequency update features are obtained by convolving the high-frequency features and interpolating the low-frequency features using the following formulas, and the low-frequency update features are obtained by convolving the low-frequency features and performing average pooling on the high-frequency features:
[0096] Y H ={Y H→H +Y L→H},Y L ={Y L→L +Y H→L}
[0097] Y H =f(X) H W H→H )+upsample(f(X L W L→H ),2)
[0098] Y L =f(X) L W L→L )+f(poll(X H ,2);W H→L )
[0099] Where f(X; W) represents a convolution with parameters W, poll(X, k) is an average pooling operation with a kernel size of k×k and a stride of k, and upsample(X, k) is an upsampling operation performed by factor k using the most recent interpolation. A→BW represents the convolutional update from feature map group A to group B. A→B This represents the corresponding convolution kernel.
[0100] See below Figure 4 The above embodiments are illustrated by way of example.
[0101] 1. First, convolve the low-resolution image to obtain image features, R X ∈R c×h×w .
[0102] 2. Feature extraction is performed based on the three DCMs to obtain D1, D2, and D3. This is then applied to R... X D1, D2, and D3 are concatenated and then subjected to a 1×1 convolution operation to obtain R. fusion =Conv (4*c,1,c) [R X [D1,D2,D3]
[0103] 3. R fusion =Conv (4*c,1,c) [R X An octave convolution operation is performed on [D1,D2,D3] to obtain a high-resolution image.
[0104] Based on the above embodiments, this embodiment provides an application example.
[0105] Step 1: Construct training data.
[0106] We applied the second-order degradation model from the Real-ESRGAN paper to degrade the original images, generating LR-HR image pairs for training. This degradation model performs a two-stage degradation process, each stage comprising four steps: Blur, Resize, Noise, and JPEG Compression. Each step incorporates its own classic model, randomly selected from among them. In the final degradation stage, the model also uses a sinc filter to synthesize common ringing and overshoot artifacts, thus better covering the degradation conditions of real-world remote sensing images.
[0107] Step 2: Dynamic Multi-Scale Filter Module
[0108] Benefiting from ESRGAN's excellent performance in natural image super-resolution tasks, we use it as the generator network for super-resolution tasks in the petroleum industry. The constructed training data is fed into the ESRGAN network for training. To better adapt to the complex spatial distribution and large object size variations in petroleum industry images, inspired by DMNet, we introduce a DCM module into the ESRGAN network, enabling the generator network to dynamically capture multi-scale features. Figure 4As shown, let the input feature map be R. X ∈R c×h×w Where h, w, and c are the height, width, and number of channels, respectively. We used three DCM modules to perform feature extraction on Rx in parallel to obtain D1∈R. c×h×w D2∈R c×h×w ,D3∈R c×h×w DCM is a context-aware filter with convolutional kernel sizes of 1×1, 3×3, and 5×5. Then we apply this to R... X D1, D2, and D3 are concatenated, and the concatenated features are further integrated through a convolutional layer, thereby effectively fusing the captured dynamic multi-scale contextual information.
[0109] R fusion =Conv (4*c,1,c) [R X ,D1,D2,D (1)
[0110] Where Conv (i,k,o) It is a k×k convolutional layer, where i is the number of input channels and o is the number of output channels.
[0111] The principle of DCM is as follows Figure 2 As shown, we first use convolutional layers to simplify the input to obtain a reduced feature map.
[0112]
[0113] Then, we extract contextual features through adaptive average pooling and 1×1 convolution, and then assign corresponding weights to the context-aware filter gk(x) to enable the network structure to adapt to the input image.
[0114]
[0115] Next, we will use the obtained context-aware filter and the reduced feature map R reduce Perform convolution to obtain
[0116]
[0117] Finally, 1×1 convolutions are applied to fuse the channel information, resulting in a specific output.
[0118]
[0119] Step 3: Octal Convolution Structure
[0120] Because a second-order degenerate model is used during training, resulting in a large degradation space, training the ESRGAN network becomes difficult. Therefore, we need to improve the discriminator's ability to accurately distinguish between generated and real images, providing more accurate gradient feedback and thus enhancing the quality of generated images. Considering that the discriminator is essentially a classifier, we can explore effective improvements to the classification network to enhance its performance.
[0121] Inspired by octave convolution, we introduced an octave convolution structure into the discriminator. For example... Figure 3 As shown, we replace traditional convolution with octave convolution and apply it to the first eight layers of the discriminator network. The frequency-doubling convolution is inspired by frequency domain transformation in traditional image processing. It is assumed that feature maps also have corresponding frequency domain maps, which can be divided into low-frequency and high-frequency components, thus yielding a new multi-frequency feature representation method. Specifically, high-frequency and low-frequency features can be mapped and stored in different groups. Clearly, there is significant redundancy in the low-frequency components; therefore, we halve it, as shown below:
[0122] X = {X H ,X L} (6)
[0123] In the formula, X∈R c×h×w Let X represent the input feature tensor. H ∈R (1-α)×c×h×w X represents the high-frequency feature map after decomposition. L ∈R α×c×h×w This represents the decomposed low-frequency feature map, where h, w, and c are the height, width, and number of channels, respectively, and α represents the ratio of channels allocated to the low-frequency part, which we set to half.
[0124] Because the resolution of low-frequency feature maps decreases, the receptive field of octave convolutions increases. Therefore, when processing low-frequency feature maps using convolutional kernels, the receptive field of octave convolutions is almost twice that of ordinary convolutions. This further helps octave convolutional layers capture long-range contextual information, while the interaction between high and low frequencies facilitates the utilization of multi-scale information. These improvements ultimately enhance the discriminator's performance. We introduced octave convolutions into the discriminator, enabling it to achieve multi-scale feature fusion capabilities at the convolutional level.
[0125] The formula for the exchange between high- and low-frequency feature maps is as follows:
[0126] Y H ={Y H→H +Y L→H},Y L ={Y L→L +Y H→L} (7)
[0127] Y L =f(X) L W L→L )+f(poll(X H ,2);W H→L ) (8)
[0129] In the formula, f(X; W) represents a convolution with parameter W, poll(X, k) is an average pooling operation with kernel size k×k and stride k, and upsample(X, k) is an upsampling operation performed by factor k through the most recent interpolation. A→B W represents the convolutional update from feature map group A to group B. A→B This represents the corresponding convolution kernel.
[0130] Step 4: Super-resolution reconstruction
[0131] After training, the trained network is used for testing. The input second image is subjected to image super-resolution to obtain a super-resolution image with the corresponding super-resolution factor.
[0132] Based on the above embodiments, this embodiment provides a network training device and a data acquisition module for acquiring a training dataset. The training dataset includes a first image and a second image, wherein the second image is acquired based on the first image, and the resolution of the second image is lower than the resolution of the first image.
[0133] The feature extraction module is used to input the second image into the network, extract features from the second image, and obtain multi-scale image features, wherein the multi-scale image features are generated based on image features of different scales;
[0134] The processing module is used to delete and reconstruct redundant information in the multi-scale image features to obtain a third image corresponding to the second image;
[0135] A training module is used to train the network based on the differences between the first image and the third image.
[0136] In some embodiments, inputting the second image into the network and extracting features from the second image to obtain multi-scale image features includes:
[0137] The second image is input into the network, and features are extracted from the second image based on multiple scales to obtain multiple types of image features, wherein each type of image feature corresponds to a scale;
[0138] The multi-scale image features are obtained by concatenating the various image features and integrating them through convolution.
[0139] In some embodiments, the step of inputting the second image into the network and extracting features from the second image based on multiple scales to obtain multiple types of image features includes:
[0140] The second image is input into the network, and the second image is reduced to obtain the reduction feature;
[0141] The second image is subjected to adaptive averaging, convolution, and weight assignment to obtain contextual features.
[0142] The image features are obtained by convolution and fusion of the reduced features and the context features.
[0143] In some embodiments, the step of deleting and reconstructing redundant information in the multi-scale image features to obtain a third image corresponding to the second image includes:
[0144] The redundant information of the multi-scale image features is removed based on the octave convolution structure, and the third image is reconstructed.
[0145] In some embodiments, each degree of the octave convolutional structure includes:
[0146] The multi-scale image features are decomposed to obtain high-frequency features and low-frequency features;
[0147] The high-frequency features are convolved and the low-frequency features are interpolated to obtain the high-frequency updated features;
[0148] The low-frequency features are convolved and the high-frequency features are averaged pooled to obtain the low-frequency updated features.
[0149] In some embodiments, the method further includes:
[0150] The first image is blurred to obtain a blurred image;
[0151] The blurred image is scaled to obtain a scaled image;
[0152] The scaled image is denoised to obtain a denoised image;
[0153] The denoised image is compressed to obtain the second image.
[0154] In some embodiments, the method further includes:
[0155] Based on the trained network, the fourth image is enhanced to obtain a fifth image corresponding to the fourth image, wherein the resolution of the fifth image is higher than that of the fourth image.
[0156] Based on the above embodiments, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above embodiments.
[0157] In some embodiments of this example, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the steps of the method described in the above embodiments.
[0158] In some embodiments of this example, a computer program product is provided, including a computer program / instructions, characterized in that the computer program, when executed by a processor, implements the steps of the method described in the above embodiments.
[0159] The processor may include, but is not limited to, one or more processors or microprocessors. Each processor may be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic component, for executing the methods in the above embodiments.
[0160] Computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof. Computer-readable storage media may include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, and computer storage media (e.g., hard disks, floppy disks, solid-state drives, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, etc.).
[0161] Computer-readable storage media may also store at least one computer-executable program / instruction, such as computer-readable instructions. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above can be performed.
[0162] In addition, the computer device may include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (e.g., keyboard, mouse, speakers, etc.).
[0163] The processor can communicate with external devices via the I / O bus through wired or wireless networks.
[0164] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.
[0165] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0166] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0167] While the embodiments disclosed herein are as described above, the foregoing content is merely for the purpose of facilitating understanding of this disclosure and is not intended to limit this disclosure. Any person skilled in the art to which this disclosure pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope of this disclosure; however, the scope of patent protection of this disclosure shall still be determined by the scope defined in the appended claims.
Claims
1. A network training method, characterized in that, include: Obtain a training dataset, which includes a first image and a second image, wherein the second image is obtained based on the first image and the resolution of the second image is lower than that of the first image; The second image is input into the network, and features are extracted from the second image to obtain multi-scale image features, wherein the multi-scale image features are generated based on image features at different scales; Redundant information in the multi-scale image features is deleted and reconstructed to obtain a third image corresponding to the second image; The network is trained based on the differences between the first image and the third image.
2. The method according to claim 1, characterized in that, The step of inputting the second image into the network and extracting features from the second image to obtain multi-scale image features includes: The second image is input into the network, and features are extracted from the second image based on multiple scales to obtain multiple types of image features, wherein each type of image feature corresponds to a scale; The multi-scale image features are obtained by concatenating the various image features and integrating them through convolution.
3. The method according to claim 2, characterized in that, The second image is input into the network, and features are extracted from the second image based on multiple scales to obtain multiple types of image features, including: The second image is input into the network, and the second image is reduced to obtain the reduction feature; The second image is subjected to adaptive averaging, convolution, and weight assignment to obtain contextual features. The image features are obtained by convolution and fusion of the reduced features and the context features.
4. The method according to claim 1, characterized in that, The process of deleting redundant information and reconstructing the multi-scale image features to obtain the third image corresponding to the second image includes: The redundant information of the multi-scale image features is removed based on the octave convolution structure, and the third image is reconstructed.
5. The method according to claim 4, characterized in that, Each degree of the octave convolutional structure includes: The multi-scale image features are decomposed to obtain high-frequency features and low-frequency features; The high-frequency features are convolved and the low-frequency features are interpolated to obtain the high-frequency updated features; The low-frequency features are convolved and the high-frequency features are averaged pooled to obtain the low-frequency updated features.
6. The method according to claim 1, characterized in that, The method further includes: The first image is blurred to obtain a blurred image; The blurred image is scaled to obtain a scaled image; The scaled image is denoised to obtain a denoised image; The denoised image is compressed to obtain the second image.
7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Based on the trained network, the fourth image is enhanced to obtain a fifth image corresponding to the fourth image, wherein the resolution of the fifth image is higher than that of the fourth image.
8. A network training device, characterized in that, include: The data acquisition module is used to acquire a training dataset, which includes a first image and a second image, wherein the second image is acquired based on the first image and the resolution of the second image is lower than that of the first image; The feature extraction module is used to input the second image into the network, extract features from the second image, and obtain multi-scale image features, wherein the multi-scale image features are generated based on image features of different scales; The processing module is used to delete and reconstruct redundant information in the multi-scale image features to obtain a third image corresponding to the second image; A training module is used to train the network based on the differences between the first image and the third image.
9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7.
11. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7.