Rapid image denoising method in regional space, medium, equipment and application

Abstract

The invention relates to a rapid image denoising method in a regional space, a medium, equipment and application. The method comprises the following steps: collecting standardized samples, and constructing a standardized sample set; constructing a lightweight denoising network; inputting the continuous noise samples into a lightweight denoising network, and training the lightweight denoising network in cooperation with the standardized samples; inputting a real-time continuous noise image into the trained lightweight de-noising network, outputting a de-noised image and refreshing a de-noised data set; implementing hardware by using the method; the method is applied to rapid image denoising and VR equipment positioning in a VR scene. According to the method, a double-branch lightweight denoising network is established, space redundancy is reduced, the model efficiency is high, and the image denoising speed is high; the features are better fused by obtaining the probability distribution of the features, and then the image is denoised and restored; network parameters are few, training is fast, and the method is especially suitable for small-range scenes or areas; when the method is applied to a VR scene, rapid image denoising can be achieved, and rapid positioning and repositioning of VR equipment can be achieved in a matched mode.

Description

Technical Field

[0001] The present invention relates to the technical field of general image data processing or generation, and particularly relates to a method, medium, device and application for fast image denoising in a regional space. Background Art

[0002] Image noise refers to unnecessary or redundant interference information existing in image data. During the generation and transmission of images, the image quality often deteriorates due to various noise interferences and influences. The existence of noise seriously affects the image quality, but it is very difficult to completely avoid. Therefore, it is necessary to perform denoising processing on the image before further utilization.

[0003] The main methods of image denoising include filtering based on filters or denoising after learning the features and noise distribution of the image based on deep learning techniques, etc.

[0004] The denoising effect of filtering based on filters and building models is obvious to all. Although such denoising methods have achieved good denoising effects, their defects are also obvious, including but not limited to loss of image details, noise amplification, high design complexity, possible loss of some important information of the image during the denoising process, and the need to optimize the filter for specific noise types, etc.

[0005] Therefore, more and more researchers choose neural network models for image denoising. However, the neural networks in the prior art often have complex structures, large numbers of parameters, and complex training. This may be necessary for large-scale application scenarios, but in fact, for scenarios with a certain or small regional space, such large-scale and deep neural networks are not applicable.

[0006] Furthermore, considering the scenarios with a certain or small regional space, in fact, the collected images are homogeneous images with many similar features. If this can be fully utilized, the burden on the neural network can be greatly reduced, and training and utilization can be quickly achieved. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides a method, medium, device and application for fast image denoising in a regional space.

[0008] The technical solution adopted by the present invention is a method for fast image denoising in a regional space, which includes collecting standardized samples and constructing a standardized sample set; constructing a lightweight denoising network; inputting continuous noise samples into the lightweight denoising network, and training the lightweight denoising network in cooperation with the standardized samples; Inputting a real-time continuous noise image into the trained lightweight denoising network, outputting a denoised image and refreshing the denoising data set.

[0009] Preferably, the standardized sample includes a plurality of images within the same regional space. Data augmentation is performed on the standardized sample to establish a standardized sample set.

[0010] Preferably, feature processing and labeling are performed on the sample data in the standardized sample set to form an image-feature-label pair.

[0011] Preferably, the lightweight denoising network includes a teacher-student double-branch structure; The student branch corresponding to the continuous noise sample includes a feature extraction layer, a feature enhancement layer, a feature fusion layer, and a ResNet layer arranged in sequence. The input of the student branch is added to the output of the ResNet layer and then output.

[0012] Preferably, the teacher branch corresponding to the standardized sample includes a sample extraction module that cooperates with the feature extraction layer and is used to obtain standardized samples with different probability distributions; A feature extraction block based on probability distribution is sequentially connected after the sample extraction module. The feature extraction block based on probability distribution is respectively associated with the feature enhancement layer and the feature fusion layer; The feature extraction block is connected to the ResNet layer after that.

[0013] Preferably, the loss function L is associated with the output loss L1 of the double-branch structure and the independent output loss L2 of the student branch.

[0014] Preferably, the signal-to-noise ratio change rate between the newly stored image and the previous image in the denoising dataset is calculated at a preset frequency. If it is greater than the preset value, the denoising dataset is cleared and the lightweight denoising network is retrained.

[0015] A computer-readable storage medium stores a fast image denoising program in the regional space. When the program is executed by a processor, the fast image denoising method in the regional space as described above is implemented.

[0016] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the fast image denoising method in the regional space as described above is implemented.

[0017] An application of the fast image denoising method in the regional space as described above is applied to fast image denoising in the VR scenario and VR device positioning.

[0018] The present invention relates to a method, medium, device and application for fast image denoising in a regional space, which includes collecting standardized samples to construct a standardized sample set; constructing a lightweight denoising network; inputting continuous noise samples into the lightweight denoising network and training the lightweight denoising network in cooperation with the standardized samples; inputting real-time continuous noise images into the trained lightweight denoising network, outputting denoised images and refreshing the denoising data set; implementing the method in hardware; and applying it to fast image denoising and VR device positioning in a VR scene.

[0019] The beneficial effects of the present invention are as follows: (1) A lightweight denoising network with a dual-branch structure is established. By utilizing the training characteristics of the teacher branch and the student branch, information is shared while obtaining feature vectors, reducing spatial redundancy, having high model efficiency, and fast image denoising speed; (2) By obtaining the probability distribution of features, features can be better fused, and then the image can be denoised and repaired; (3) The network has few parameters and fast training, and is particularly suitable for small-scale scenarios or regions; (4) When applied in a VR scene, in addition to achieving fast image denoising, it can also cooperate to achieve fast positioning and repositioning of VR devices. Description of the Drawings

[0020] Figure 1 is a flowchart of the present invention; Figure 2 is a schematic structural diagram of the lightweight denoising network of the present invention. Detailed Embodiments

[0021] The following further describes the present invention in detail with reference to embodiments, but the protection scope of the present invention is not limited thereto.

[0022] The present invention relates to a method for fast image denoising in a regional space, which specifically includes the following steps: (1) Collect standardized samples to construct a standardized sample set; (2) Construct a lightweight denoising network; (3) Input continuous noise samples into the lightweight denoising network and train the lightweight denoising network in cooperation with the standardized samples; (4) Input real-time continuous noise images into the trained lightweight denoising network, output denoised images and refresh the denoising data set.

[0023] The following describes in combination with specific steps.

[0024] (1) Collect standardized samples to construct a standardized sample set; The standardized samples include several images in the same regional space. Data augmentation is performed on the standardized samples to establish a standardized sample set.

[0025] Perform feature processing and labeling on the sample data in the standardized sample set to form image-feature-label pairs.

[0026] In the present invention, first, the method is limited to a certain regional space. Taking the regional space in the VR scenario as an example, its range is relatively small. When setting several image acquisition angles, the feature points included in the acquired images are relatively unified, so a standardized sample set can be established.

[0027] Specifically, collect several images in the same regional space. For example, evenly divide the 360° of the horizontal projection plane into 6 parts, that is, collect multiple photos at this perspective every 60° rotation. In order to cover as many details as possible, the photos can include but are not limited to stretched pictures collected by a fish-eye camera, wide-width pictures collected by a wide-angle camera, or imaging pictures of an ordinary camera; data augmentation for the standardized samples includes cropping, rotating, distorting, etc., to fit the picture effects in various states of continuous noise sample generation.

[0028] In the present invention, for the convenience of training, manually extract features from these sample data and perform regularization labeling.

[0029] (2) Construct a lightweight denoising network; The lightweight denoising network includes a teacher-student double-branch structure; The student branch corresponding to the continuous noise samples includes a feature extraction layer, a feature enhancement layer, a feature fusion layer, and a ResNet layer arranged in sequence, and the input of the student branch is added to the output of the ResNet layer and then output.

[0030] The teacher branch corresponding to the standardized samples includes a sample extraction module that cooperates with the feature extraction layer to obtain standardized samples with different probability distributions; After the sample extraction module, a feature extraction block based on probability distribution is connected in sequence, and the feature extraction block based on probability distribution is respectively associated with the feature enhancement layer and the feature fusion layer; After the feature extraction block, it is connected to the ResNet layer.

[0031] In the present invention, using the standardized samples as the output of the teacher branch, during the subsequent training process, the parameters of the teacher branch are frozen; however, in order to better identify features, the teacher branch can be trained first.

[0032] In the present invention, continuous noise samples are input into the student branch. The reason for using continuous noise samples instead of independent images is that in the actual application process, images have continuity. In fact, during the denoising process of the subsequent image, the denoising of the previous image should also be considered. Here, the above-mentioned context attention module is applied to the sample extraction module of the teacher branch; In the student branch, through the input of continuous noise samples into the feature extraction layer, it can obtain multiple features corresponding to different samples. The feature enhancement layer enhances the features based on weight allocation, mainly enhancing the context-related features, providing more local features, and performing feature fusion with the feature fusion layer. The fused features are reconstructed with the input Figure 1 starting point.

[0033] In the present invention, the teacher branch guides the student branch throughout the process; Specifically, it realizes the sample extraction of the sample extraction module through the feature extraction layer and the cooperation of the context attention module. The sample extraction here refers to obtaining standardized samples with different probability distributions, which are used to better correspond to the current image details at the student branch and conduct subsequent reconstruction guidance; Subsequently, the feature extraction block based on the probability distribution extracts the features in different images, processes them based on the probability distribution, obtains a series of feature vectors, and then outputs and associates them with the feature enhancement layer and the feature fusion layer respectively for guidance. That is, the features of the feature enhancement layer are enhanced based on the probability distribution of the features, and the fusion of the feature fusion layer is guided by the standardized samples based on the probability distribution.

[0034] (3) Input continuous noise samples into the lightweight denoising network, and train the lightweight denoising network with the standardized samples; The loss function L is associated with the output loss L1 of the dual-branch structure and the independent output loss L2 of the student branch.

[0035] In the present invention, in fact, the teacher branch should be trained first to construct the loss function Lα, which is associated with the difference between the output and the input of the teacher branch.

[0036] In the present invention, subsequently, a loss function is established with the sum of the output loss L1 of the dual-branch structure and the independent output loss L2 of the student branch, and the network parameters of the student branch are adjusted until L is minimized. Here, the output loss L1 of the dual-branch structure refers to the reciprocal of the similarity between the output images X 2 and X 1 of the teacher branch and the student branch, and the independent output loss L2 of the student branch refers to the difference between the input X 0 and the output image X 1 of the student branch.

[0037] (4) Input the real-time continuous noise image into the trained lightweight denoising network, output the denoised image, and refresh the denoising dataset.

[0038] Calculate the signal-to-noise ratio change rate between the newly stored image and the previous image in the denoising dataset at a preset frequency. If it is greater than the preset value, clear the denoising dataset and retrain the lightweight denoising network.

[0039] In the present invention, since the denoising network is lightweight, the number of its training times can be reduced, and it is triggered by the SNR change rate of the denoised image. Generally speaking, the signal-to-noise ratio is preferably relatively high. If the rate of its decrease is greater than the preset value, retraining is required.

[0040] In the present invention, a certain amount of denoised images are stored and then released.

[0041] The present invention also relates to a computer-readable storage medium, on which a fast image denoising program in a regional space is stored. When the program is executed by a processor, the fast image denoising method in the regional space as described above is implemented.

[0042] The present invention also relates to a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the fast image denoising method in the regional space as described above is implemented.

[0043] The present invention also relates to an application of the fast image denoising method in the regional space as described above, which is applied to fast image denoising in a VR scenario and VR device positioning.

[0044] The present invention is particularly applicable to a certain range of occasions, such as in a VR scenario. By quickly denoising the image, it is possible to ensure the quick positioning of VR devices, including but not limited to the position of the handle, etc., so that users can enter the VR experience more efficiently.

[0045] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0046] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0047] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0048] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0049] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0050] Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for fast image denoising in a regional space, characterized by: Collect standardized samples and build a standardized sample set; build a lightweight denoising network; Inputting continuous noise samples into the lightweight denoising network, and training the lightweight denoising network with the standardized samples; The real-time continuous noisy image is input into the trained lightweight denoising network, the denoised image is output and the denoising dataset is refreshed.

2. The method for fast image denoising in regional space according to claim 1, characterized in that: The standardized samples include a number of images in the same area space, and data enhancement is performed on the standardized samples to establish a standardized sample set.

3. The method for fast image denoising in regional space according to claim 2, characterized in that: The sample data in the standardized sample set is feature processed and labeled to form image-feature-label pairs.

4. The method for fast image denoising in regional space according to claim 1, characterized in that: The lightweight denoising network includes a teacher-student dual-branch structure; The student branch corresponding to the continuous noise sample includes a feature extraction layer, a feature enhancement layer, a feature fusion layer and a ResNet layer which are arranged in sequence, and the input of the student branch is added to the output of the ResNet layer and then outputted.

5. The method for fast image denoising in regional space according to claim 4, characterized in that: The teacher branch corresponding to the standardized samples includes a sample extraction module cooperating with the feature extraction layer to obtain standardized samples of different probability distributions; The sample extraction module is sequentially connected to a feature extraction block based on probability distribution, and the feature extraction block based on probability distribution is associated with a feature enhancement layer and a feature fusion layer respectively; The feature extraction block is then connected to the ResNet layer.

6. The method for fast image denoising in regional space according to claim 5, characterized in that: The loss function L is associated with the output loss L1 of the dual-branch structure and the independent output loss L2 of the student branch.

7. The method for fast image denoising in regional space according to claim 1, characterized in that: The signal-to-noise ratio change rate of the newly stored image and the previous image in the denoising data set is calculated at a preset frequency. If it is greater than a preset value, the denoising data set is cleared and the lightweight denoising network is retrained.

8. A computer-readable storage medium, characterized in that: A fast image denoising program in regional space is stored thereon, and when the program is executed by a processor, the fast image denoising method in regional space described in one of claims 1 to 7 is implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for fast image denoising in regional space as described in any one of claims 1 to 7 is implemented.

10. An application of the method for fast image denoising in regional space according to any one of claims 1 to 7, characterized in that: Applied to fast image denoising and VR device positioning in VR scenarios.

Images