Solar spectrum image inversion method based on deep learning

Abstract

The invention relates to the technical field of solar spectrum inversion, and discloses a solar spectrum image inversion method based on deep learning, and the method comprises the steps: obtaining a solar spectrum image, and generating a plurality of groups of convolution kernels based on the solar spectrum image through a point spread function; fusing the solar spectrum images containing the target spectral lines by using a plurality of groups of convolution kernels to obtain a multi-spectral-line mixed solar spectrum simulation data set; and performing multi-layer convolution on the multi-spectral-line mixed solar spectrum simulation data set to obtain a solar spectrum image inversion result. According to the method, a solar spectrum image inversion result is obtained; the obtained multi-spectral-line mixed solar spectrum simulation data set retains image details, inhibits irrelevant noise in the image, realizes inversion of the solar spectrum image, and enhances the inversion effect.

Description

Technical Field

[0001] The present invention relates to the technical field of solar spectrum inversion, and in particular to a solar spectrum image inversion method based on deep learning. Background Art

[0002] The solar spectrum is an absorption spectrum of different wavelengths, which has the physical characteristics of relatively uniform energy distribution of each spectral line and large imaging interference between adjacent spectral lines. In the field of solar spectrum inversion, the main bottleneck of using deep learning algorithms with better performance lies in the construction of noisy-noiseless image pair datasets. Among them, the spectral images of specific spectral lines can be directly obtained, and the simulation synthesis of noisy images (i.e., multi-spectral mixed spectral images) mainly relies on the method of directly adding noise or using complex generative adversarial networks.

[0003] However, in the simulation process of the above-mentioned noisy image multi-spectral mixed spectral image, there are problems such as poor image detail preservation and unsatisfactory irrelevant noise suppression effect, resulting in poor simulation image synthesis effect. Summary of the invention

[0004] In view of this, the present invention provides a solar spectrum image inversion method based on deep learning to solve the problems of poor image detail preservation and unsatisfactory irrelevant noise suppression effect in the simulation process of noisy image multi-spectral mixed spectrum images.

[0005] In a first aspect, the present invention provides a solar spectrum image inversion method based on deep learning, the method comprising:

[0006] Acquire a solar spectrum image, and generate multiple groups of convolution kernels using a point spread function based on the solar spectrum image;

[0007] Multiple groups of convolution kernels are used to fuse solar spectrum images containing target spectral lines to obtain a multi-spectral mixed solar spectrum simulation data set;

[0008] Multi-layer convolution is performed on the multi-spectral mixed solar spectrum simulation data set to obtain the solar spectrum image inversion result.

[0009] The deep learning-based solar spectrum image inversion method provided in this embodiment uses a solar spectrum image to generate multiple groups of convolution kernels using a point spread function, and then uses the multiple groups of convolution kernels to fuse the solar spectrum image containing the target spectrum line to obtain a multi-spectral mixed solar spectrum simulation data set, which significantly improves the synthesis effect of the simulation image, can effectively retain the detail information in the solar spectrum image, suppress the irrelevant noise in the image, and realize the efficient simulation of the multi-band mixed spectrum image; then, multi-layer convolution is performed on the multi-spectral mixed solar spectrum simulation data set to obtain the solar spectrum image inversion result, thereby realizing the inversion of the solar spectrum image and enhancing the inversion effect.

[0010] In an optional implementation, based on the solar spectrum image, multiple groups of convolution kernels are generated using a point spread function, including:

[0011] The target spectrum line point light source corresponding to the solar spectrum image is input into the physical optical system for testing to obtain the point spread function of the target spectrum line;

[0012] A deep convolutional neural network is obtained, and the point spread function of the target spectral line is fitted using the convolution kernel parameters corresponding to the deep convolutional neural network to obtain multiple groups of convolution kernels.

[0013] The solar spectrum image inversion method based on deep learning provided in this embodiment is implemented by inputting the target spectrum line point light source corresponding to the solar spectrum image into the physical optical system for testing, so as to obtain the point spread function of the target spectrum line, and then obtain the deep convolutional neural network, and fit the point spread function of the target spectrum line using the convolution kernel parameters corresponding to the deep convolutional neural network to obtain multiple groups of convolution kernels; in the process of fitting the point spread function of the target spectrum line, the simulation of various aberrations, diffraction effects and other physical influencing factors of real light in the optical system is realized, the image details are effectively retained, and the simulation effect close to the actual imaging is obtained.

[0014] In an optional implementation, based on the solar spectrum image, multiple groups of convolution kernels are generated using a point spread function, further comprising:

[0015] The number of convolution kernel additions of the target spectral line is obtained, and the number of convolution kernel additions of the target spectral line is added to the multiple groups of convolution kernels.

[0016] The deep learning-based solar spectrum image inversion method provided in this embodiment obtains the number of convolution kernel additions for the target spectrum line, adds the number of convolution kernel additions for the target spectrum line to multiple groups of convolution kernels, and further increases the number of convolution kernels acting on specific spectrum lines, thereby achieving refined retention of details in key spectrum line images and laying a foundation for solar spectrum image fusion.

[0017] In an optional implementation, multiple groups of convolution kernels are used to fuse solar spectrum images containing target spectrum lines to obtain a multi-spectral mixed solar spectrum simulation data set, including:

[0018] Convolution channels are respectively set for a plurality of target spectral lines corresponding to the spectral images of the target spectral lines;

[0019] The convolution kernel is used to perform a convolution operation on the spectral image of the target spectral line after setting the convolution channel to obtain the intermediate features;

[0020] The intermediate features are linearly superimposed to obtain a multi-spectral mixed solar spectrum simulation data set.

[0021] The solar spectrum image inversion method based on deep learning provided in this embodiment sets convolution channels for multiple target spectral lines corresponding to the spectral images of the target spectral lines respectively, and then uses the convolution kernel to perform a convolution operation on the spectral images of the target spectral lines after setting the convolution channels to obtain intermediate features, and finally linearly superimposes the intermediate features to obtain a multi-spectral mixed solar spectrum simulation data set; based on multi-channel convolution, it fully simulates the phenomenon of spectral interference between different target spectral lines, improves the quality of the simulated image, can effectively retain the detail information in the image, and realizes efficient multi-band mixed spectrum image simulation, so that the multi-spectral mixed solar spectrum simulation data set can support the training of deep learning algorithms and achieve better performance.

[0022] In an optional implementation, multi-layer convolution is performed on the multi-spectral mixed solar spectrum simulation data set to obtain a solar spectrum image inversion result, including:

[0023] Based on the multi-spectral mixed solar spectrum simulation dataset, a deep convolutional neural network is used to extract the semantic information and texture information in the solar spectrum image, and the solar spectrum image inversion result is determined based on the semantic information and texture information.

[0024] The solar spectrum image inversion method based on deep learning provided in this embodiment extracts semantic information and texture information from the solar spectrum image based on a multi-spectral mixed solar spectrum simulation data set using a deep convolutional neural network, and determines the solar spectrum image inversion result based on the semantic information and texture information; in the image inversion process, the semantic information and texture information can be effectively utilized, laying the foundation for reconstructing or inferring the original image information.

[0025] In an optional embodiment, based on a multi-spectral mixed solar spectrum simulation data set, using a deep convolutional neural network to extract semantic information and texture information in a solar spectrum image, and determining a solar spectrum image inversion result based on the semantic information and texture information also includes:

[0026] The multi-spectral mixed solar spectrum simulation dataset is used as input data, and supervised learning is performed on the output data of the deep convolutional neural network to obtain the trained deep convolutional neural network.

[0027] The deep learning-based solar spectrum image inversion method provided in this embodiment uses a multi-spectral mixed solar spectrum simulation data set to perform supervised learning on the output data of a deep convolutional neural network, which can support the training of a deep convolutional neural network and achieve better performance. In addition, it reduces the investment in human labeling during the training process and improves the image inversion efficiency.

[0028] In a second aspect, the present invention provides a solar spectrum image inversion device based on deep learning, the device comprising:

[0029] A generation module, used for acquiring a solar spectrum image, and generating multiple groups of convolution kernels based on the solar spectrum image by using a point spread function;

[0030] A fusion module is used to fuse solar spectrum images containing target spectrum lines using multiple groups of convolution kernels to obtain a multi-spectral mixed solar spectrum simulation data set;

[0031] The convolution module is used to perform multi-layer convolution on the multi-spectral mixed solar spectrum simulation data set to obtain the solar spectrum image inversion result.

[0032] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the deep learning-based solar spectrum image inversion method of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0033] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the solar spectrum image inversion method based on deep learning according to the first aspect or any corresponding embodiment thereof.

[0034] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the deep learning-based solar spectrum image inversion method of the above-mentioned first aspect or any corresponding embodiment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 is a schematic flow chart of a solar spectrum image inversion method based on deep learning according to an embodiment of the present invention;

[0037] Figure 2 is a flow chart of another solar spectrum image inversion method based on deep learning according to an embodiment of the present invention;

[0038] Figure 3 is a flow chart of another solar spectrum image inversion method based on deep learning according to an embodiment of the present invention;

[0039] Figure 4is a flow chart of a method for fusing spectral images of different spectral lines based on multi-channel convolution according to an embodiment of the present invention;

[0040] Figure 5 is a flow chart of spectral image inversion based on U-Net according to an embodiment of the present invention;

[0041] Figure 6 is a structural block diagram of a solar spectrum image inversion device based on deep learning according to an embodiment of the present invention;

[0042] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0044] Inversion is a basic task in the field of image processing. The related inversion algorithms have no special requirements for the input data set, while the deep learning inversion algorithms with better performance require a large number of noisy-noiseless image pairs as input, but it is extremely difficult to obtain data for such images. Especially in the scenario of using space devices to collect multi-spectral mixed spectral images, the cost of obtaining multi-spectral mixed spectral images and the corresponding separate imaging of each spectral line is too high. Therefore, in this scenario, the construction of the training data set becomes a key bottleneck restricting the deep learning algorithms with better performance.

[0045] The solar spectrum is presented as a continuous spectrum, which has the physical characteristics of relatively uniform energy distribution of each spectral line and large imaging interference between adjacent spectral lines. At the same time, due to the extremely high timeliness of spectral images, it is difficult to simultaneously collect mixed spectra and separate imaging of each spectral line by hardware means. The point spread function can describe the imaging distribution form of an ideal point light source on a sensor or receiver after passing through an optical system. Based on this function, the influence of various aberrations, diffraction effects and other physical factors in the optical system can be modeled from a mathematical perspective. Using the point spread function as a guide for simulation synthesis can obtain a simulation effect that is closer to actual imaging.

[0046] Multi-channel convolution can extract features from spectral images from different sources and linearly superimpose them. It can also flexibly adjust the convolution parameters to achieve a good fit to the point spread function. Therefore, it can better simulate and synthesize noisy spectral images with multiple spectral lines, effectively overcoming the problem that such noisy images are difficult to obtain in the data set. U-Net is a deep convolutional neural network, mainly composed of convolution modules and downsampling modules. It can effectively extract features of different scales in images and is often used for image inversion denoising, supersampling and segmentation tasks.

[0047] In order to obtain a considerable-scale data set for training deep learning algorithms, it is first necessary to synthesize noisy simulated images based on noise-free clear images. Directly adding Gaussian noise, salt and pepper noise, and other forms of image noise or training Generative Adversarial Networks (GAN) can be used to obtain noisy images. However, for multi-line mixed spectral images, the solution of directly adding a certain form of noise to synthesize noisy images is difficult to simulate the mutual influence between different spectral lines, and will destroy the original details in the image, making it difficult to obtain high-quality noisy simulated images. Although the noisy image synthesis method based on GAN can output high-quality simulated images, it is only applicable to situations where a large number of noisy-noise-free image pairs have been obtained, and is not suitable for the needs of obtaining a large number of multi-line mixed spectral images based on specific spectral line spectral images in this scenario.

[0048] In the field of solar spectrum inversion, the main bottleneck of using deep learning algorithms with better performance lies in the construction of noisy-noiseless image pair datasets. Among them, spectral images of specific spectral lines can be directly obtained, and the simulation synthesis of noisy images (i.e., multi-spectral mixed spectral images) mainly relies on the method of directly adding noise or using complex generative adversarial networks. However, when using multi-spectral mixed spectral images for simulation, the method of directly adding noise will lead to problems such as loss of image details and poor quality of simulated images; the method based on generative adversarial networks requires a large number of multi-spectral mixed spectral images, but the actual acquisition cost of this type of spectral images is high, and it conflicts with the actual acquisition demand of solar spectrum scenes that need to reduce this type of images. Therefore, it is impossible to find a good balance between the synthesis quality and acquisition cost of this type of spectral simulation images.

[0049] The present invention aims to solve the problem that the data set required by deep learning algorithms in the field of solar spectrum inversion is difficult to construct, especially the problems of poor detail retention and unsatisfactory irrelevant noise suppression in the simulation process of multi-spectral mixed spectral images, while expanding the versatility of the algorithm for different spectral lines.

[0050] The embodiment of the present invention provides a solar spectrum image inversion method based on deep learning. By introducing a multi-channel convolution structure and combining it with an existing calibrated point spread function, the synthetic effect of the simulated image is significantly improved, the detailed information in the image can be effectively retained, and the demand for existing data is greatly reduced, thus realizing efficient multi-band mixed spectrum image simulation. The training of the deep learning inversion algorithm is then completed on a data set that meets the acquisition scale and image quality requirements.

[0051] In order to solve the above technical problems, the embodiments of the present invention provide a solar spectrum image inversion method based on deep learning. It should be noted that the solar spectrum image inversion method based on deep learning provided by the embodiments of the present invention can be executed by a device for solar spectrum image inversion based on deep learning. The device for solar spectrum image inversion based on deep learning can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware. The electronic device can be a server or a terminal. The server in the embodiments of the present application can be a single server or a server cluster composed of multiple servers. The terminal in the embodiments of the present application can be a smart phone, a personal computer, a tablet computer, a wearable device, an intelligent robot, and other intelligent hardware devices. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0052] According to an embodiment of the present invention, an embodiment of a solar spectrum image inversion method based on deep learning is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0053] In this embodiment, a solar spectrum image inversion method based on deep learning is provided, which can be used in the above-mentioned electronic device. Figure 1 is a flow chart of a solar spectrum image inversion method based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0054] Step S101, obtaining a solar spectrum image, and generating multiple groups of convolution kernels based on the solar spectrum image using a point spread function.

[0055] Specifically, for an optical system, the light field distribution of its output image when the input object is a point light source is called the point spread function.

[0056] Step S102, using multiple groups of convolution kernels to fuse the solar spectrum images containing the target spectrum lines to obtain a multi-spectral mixed solar spectrum simulation data set.

[0057] Step S103, performing multi-layer convolution on the multi-spectral mixed solar spectrum simulation data set to obtain a solar spectrum image inversion result.

[0058] Specifically, based on a multi-spectral mixed solar spectrum simulation dataset, a deep convolutional neural network is used to extract semantic information and texture information from solar spectrum images, and the solar spectrum image inversion results are determined based on the semantic information and texture information.

[0059] Furthermore, the multi-spectral mixed solar spectrum simulation dataset is taken as input, and the semantic information and texture information in the multi-spectral mixed solar spectrum simulation dataset image is effectively extracted through multi-layer convolution in U-Net, and the inversion result is output.

[0060] Furthermore, a multi-spectral mixed solar spectrum simulation dataset is used as input data, and supervised learning is performed on the output data of the deep convolutional neural network to obtain a trained deep convolutional neural network.

[0061] Furthermore, based on a multi-spectral mixed solar spectrum simulation dataset, the output results of U-Net are supervised; after a specified number of iterations or when the inversion performance meets the index requirements, the trained U-Net is obtained.

[0062] The deep learning-based solar spectrum image inversion method provided in this embodiment sets convolution channels for multiple target spectral lines corresponding to the spectral images of the target spectral lines respectively, and then uses a convolution kernel to perform a convolution operation on the spectral images of the target spectral lines after setting the convolution channels to obtain intermediate features, and finally linearly superimposes the intermediate features to obtain a multi-spectral mixed solar spectrum simulation data set; in the process of linearly superimposing the intermediate features, the intermediate features of each channel remain independent of each other, thereby realizing refined processing of images of different spectral lines; the multi-spectral mixed solar spectrum simulation data set obtained by linearly superimposing the intermediate features realizes the simulation of multi-band mixed spectral images with different target frequency bands as the main ones, and provides a data set with scale and image quality that meets the requirements for deep learning inversion algorithm training.

[0063] In this embodiment, a solar spectrum image inversion method based on deep learning is provided, which can be used in the above-mentioned electronic device. Figure 2 is a flow chart of a solar spectrum image inversion method based on deep learning according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0064] Step S201, obtaining a solar spectrum image, and generating multiple groups of convolution kernels based on the solar spectrum image using a point spread function.

[0065] Specifically, the above step S201 includes:

[0066] Step S2011, input the target spectrum line point light source corresponding to the solar spectrum image into the physical optical system for testing to obtain the point spread function of the target spectrum line.

[0067] Specifically, a physical optical system with properties similar to those of a real image acquisition device is built, and the target spectral line point light source corresponding to the solar spectrum image is input into the physical optical system to obtain the imaging result of the target spectral line. The functional relationship between the target spectral line point light source and the imaging is calculated to obtain the point spread function; wherein the point spread function obtained by the test is a discrete distribution function, which can be expressed as an m*m matrix.

[0068] Step S2012, obtaining a deep convolutional neural network, and fitting the point spread function of the target spectral line using the convolution kernel parameters corresponding to the deep convolutional neural network to obtain multiple groups of convolution kernels.

[0069] Specifically, the point spread function is fitted by adjusting the convolution kernel parameters to simulate the various aberrations, diffraction effects and other physical factors that real light is subject to in the optical system, thereby obtaining n groups of independent convolution kernels.

[0070] Furthermore, by adding and subtracting the convolution kernel parameters of n target spectral lines, the mutual influence between multiple target spectral lines is modeled.

[0071] Furthermore, the number of convolution kernel additions for the target spectral line is obtained, and the number of convolution kernel additions for the target spectral line is added to the multiple groups of convolution kernels.

[0072] Furthermore, increasing the number of convolution kernels acting on the target spectral line can preserve the details in the solar spectrum image more finely.

[0073] Step S202: Use multiple groups of convolution kernels to fuse the solar spectrum images containing the target spectrum line to obtain a multi-spectral mixed solar spectrum simulation data set. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0074] Step S203, perform multi-layer convolution on the multi-spectral mixed solar spectrum simulation data set to obtain the solar spectrum image inversion result. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0075] The solar spectrum image inversion method based on deep learning provided in this embodiment is implemented by inputting the target spectrum line point light source corresponding to the solar spectrum image into the physical optical system for testing, so as to obtain the point spread function of the target spectrum line, and then obtain the deep convolutional neural network, and fit the point spread function of the target spectrum line using the convolution kernel parameters corresponding to the deep convolutional neural network to obtain multiple groups of convolution kernels; in the process of fitting the point spread function of the target spectrum line, the simulation of various aberrations, diffraction effects and other physical influencing factors of real light in the optical system is realized, the image details are effectively retained, and the simulation effect close to the actual imaging is obtained.

[0076] In this embodiment, a solar spectrum image inversion method based on deep learning is provided, which can be used in the above-mentioned electronic device. Figure 3 is a flow chart of a solar spectrum image inversion method based on deep learning according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0077] Step S301, obtain a solar spectrum image, and generate multiple groups of convolution kernels based on the solar spectrum image using a point spread function. Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0078] Step S302, using multiple groups of convolution kernels to fuse the solar spectrum images containing the target spectrum lines to obtain a multi-spectrum mixed solar spectrum simulation data set.

[0079] Specifically, during the fusion process of the solar spectrum image, the convolution operation involved in step S302 is accelerated in parallel using the matrix calculation unit in the hardware.

[0080] In some optional implementations, the above step S302 includes:

[0081] Step S3021, respectively setting convolution channels for multiple target spectral lines corresponding to the spectral images of the target spectral lines.

[0082] Specifically, for n target spectral lines, a dedicated convolution channel is set for each target spectral line to obtain n mutually independent convolution channels.

[0083] Step S3022, using a convolution kernel to perform a convolution operation on the spectral image of the target spectral line after setting the convolution channel, to obtain an intermediate feature.

[0084] Specifically, for the spectral images of n target spectral lines, convolution operations are performed respectively using n groups of convolution kernels to obtain n groups of intermediate features; among which, the intermediate features of each convolution channel remain independent of each other, which facilitates the refined processing of images of different target spectral lines.

[0085] Step S3023, linearly superimpose the intermediate features to obtain a multi-spectral mixed solar spectrum simulation data set.

[0086] Specifically, n groups of intermediate features are linearly superimposed to model the physical process of mutual aliasing of various spectral images in the real imaging process. The mutual independence of convolution operators of different convolution channels and their invariance in the simulation process are utilized. In the process of multi-spectral mixed solar spectrum simulation, by changing the weight coefficients of different convolution channels, the simulation of multi-band mixed spectral images with different aliasing ratios and mainly different target frequency bands is realized, and a multi-spectral mixed solar spectrum simulation data set is obtained.

[0087] Step S303, perform multi-layer convolution on the multi-spectral mixed solar spectrum simulation data set to obtain the solar spectrum image inversion result. Figure 2 Step S203 of the illustrated embodiment will not be described in detail here.

[0088] The solar spectrum image inversion method based on deep learning provided in this embodiment sets convolution channels for multiple target spectral lines corresponding to the spectral images of the target spectral lines respectively, and then uses the convolution kernel to perform a convolution operation on the spectral images of the target spectral lines after setting the convolution channels to obtain intermediate features, and finally linearly superimposes the intermediate features to obtain a multi-spectral mixed solar spectrum simulation data set; in the process of linearly superimposing the intermediate features, the intermediate features of each channel remain independent of each other, thereby realizing refined processing of images of different spectral lines; the multi-spectral mixed solar spectrum simulation data set obtained by linearly superimposing the intermediate features realizes the simulation of multi-band mixed spectral images with different target frequency bands as the main ones, and provides a data set with scale and image quality that meets the requirements for the training of the deep learning inversion algorithm.

[0089] The following is a specific example to illustrate the specific steps of the solar spectrum image inversion method based on deep learning.

[0090] Embodiment 1:

[0091] The specific steps of the solar spectrum image inversion method based on deep learning include:

[0092] Firstly, the point spread function of the target spectrum line is tested based on the point spread function modeling based on the convolution operator. The measured point spread function is fitted by adjusting the internal parameters of the convolution kernel to simulate various aberrations, diffraction effects and other physical influencing factors of real light in the optical system. Finally, n groups of independent convolution kernels are obtained.

[0093] Furthermore, if Figure 4As shown in the figure, the spectral images of different spectral lines are fused based on multi-channel convolution, and the specific steps are as follows: for n different target spectral lines, a dedicated convolution channel is set for each specific spectral line, totaling n independent channels; for the spectral images of n different target spectral lines that are input, n groups of convolution kernels are convolved separately to obtain n groups of intermediate features; the intermediate features of each channel remain independent of each other, and the n groups of intermediate features are linearly superimposed to achieve modeling of the physical process of mutual overlap of spectral line images in the real imaging process.

[0094] Finally, if Figure 5 The spectral image based on U-Net is inverted as shown, and the specific steps are: by performing multi-channel convolution on the clear image of each target spectrum (i.e., the spectral image of different spectral lines), a simulated image is obtained; then the simulated image is input into a deep convolutional neural network and multiple rounds of iterations are performed to obtain the inversion image of each target spectrum, and at the same time, based on the loss function of the clear image of each target spectrum and the simulated image, a supervision network is trained, and the inversion image of each target spectrum is corrected using the supervision network and the clear image of each target spectrum.

[0095] In the above-mentioned embodiment 1, spectral images of mixed multi-spectral lines are simulated and synthesized using spectral images of different spectral lines, and the constructed data set can support the training of deep learning algorithms and achieve better performance; and, based on multi-channel convolution, the phenomenon of mutual interference between spectra of different spectral lines is fully simulated, thereby improving the quality of the simulated image; by fitting different point spread functions and flexibly adjusting the convolution parameters and channel weights in the multi-channel convolution, it is possible to conveniently simulate different target spectral line images; wherein, in the case of only clear spectral images of specific spectral lines, based on the calibrated point spread function, a high-quality multi-spectral mixed simulated spectral image can be synthesized through multi-channel convolution, thereby constructing a larger-scale multi-spectral mixed solar energy spectrum simulation data set, which can better retain details in the simulated image and highlight the mutual influence between the spectral lines. The constructed multi-spectral mixed solar energy spectrum simulation data set is sufficient to support the training of deep learning algorithms and improve the inversion performance of solar spectrum images.

[0096] In this embodiment, a solar spectrum image inversion device based on deep learning is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0097] This embodiment provides a solar spectrum image inversion device based on deep learning, such as Figure 6 As shown, including:

[0098] The generating module 601 is used to obtain a solar spectrum image and generate multiple groups of convolution kernels based on the solar spectrum image using a point spread function.

[0099] The fusion module 602 is used to fuse the solar spectrum images containing the target spectrum lines using multiple groups of convolution kernels to obtain a multi-spectral mixed solar spectrum simulation data set.

[0100] The convolution module 603 is used to perform multi-layer convolution on the multi-spectral mixed solar spectrum simulation data set to obtain a solar spectrum image inversion result.

[0101] In some optional implementations, the generating module 601 includes:

[0102] The input unit is used to input the target spectrum line point light source corresponding to the solar spectrum image into the physical optical system for testing to obtain the point spread function of the target spectrum line.

[0103] The fitting unit is used to obtain a deep convolutional neural network, and use the convolution kernel parameters corresponding to the deep convolutional neural network to fit the point spread function of the target spectral line to obtain multiple groups of convolution kernels.

[0104] In some optional implementations, the generating module 601 further includes:

[0105] The adding subunit is used to obtain the number of convolution kernels added for the target spectral line, and add the number of convolution kernels added for the target spectral line to multiple groups of convolution kernels.

[0106] In some optional implementations, the fusion module 602 includes:

[0107] A setting unit, used to respectively set convolution channels for a plurality of target spectral lines corresponding to the spectral images of the target spectral lines;

[0108] A convolution unit, used to perform a convolution operation on the spectral image of the target spectral line after setting the convolution channel using a convolution kernel to obtain an intermediate feature;

[0109] The superposition unit is used to linearly superpose the intermediate features to obtain a multi-spectral mixed solar spectrum simulation data set.

[0110] In some optional embodiments, the convolution module 603 is specifically used to extract semantic information and texture information in the solar spectrum image based on a multi-spectral mixed solar spectrum simulation data set using a deep convolutional neural network, and determine the solar spectrum image inversion result based on the semantic information and texture information.

[0111] In some optional implementations, the convolution module 603 is further used to use the multi-spectral mixed solar spectrum simulation data set as input data, perform supervised learning on the output data of the deep convolutional neural network, and obtain a trained deep convolutional neural network.

[0112] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0113] The deep learning-based solar spectrum image inversion device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0114] The embodiment of the present invention also provides a computer device having the above Figure 6 The deep learning-based solar spectrum image inversion device shown.

[0115] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0116] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0117] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0118] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0119] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0120] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0121] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0122] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0123] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0124] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A solar spectrum image inversion method based on deep learning, characterized in that: The method comprises: Acquire a solar spectrum image, and generate multiple groups of convolution kernels using a point spread function based on the solar spectrum image; Using the multiple groups of convolution kernels to fuse solar spectrum images containing target spectrum lines, to obtain a multi-spectral mixed solar spectrum simulation data set; Multi-layer convolution is performed on the multi-spectral mixed solar spectrum simulation data set to obtain a solar spectrum image inversion result.

2. The method according to claim 1, characterized in that The method of generating multiple groups of convolution kernels based on the solar spectrum image using a point spread function includes: Inputting the target spectrum line point light source corresponding to the solar spectrum image into a physical optical system for testing to obtain a point spread function of the target spectrum line; A deep convolutional neural network is obtained, and the point spread function of the target spectral line is fitted using the convolution kernel parameters corresponding to the deep convolutional neural network to obtain the multiple groups of convolution kernels.

3. The method according to claim 2, characterized in that The method of generating multiple groups of convolution kernels based on the solar spectrum image using a point spread function also includes: The number of convolution kernel additions for the target spectral line is obtained, and the number of convolution kernel additions for the target spectral line is added to the multiple groups of convolution kernels.

4. The method according to claim 1, characterized in that: The method of using the multiple groups of convolution kernels to fuse the solar spectrum images containing the target spectrum lines to obtain a multi-spectrum mixed solar spectrum simulation data set includes: respectively setting convolution channels for a plurality of target spectral lines corresponding to the spectral images of the target spectral lines; Using the convolution kernel, a convolution operation is performed on the spectral image of the target spectral line after the convolution channel is set to obtain an intermediate feature; The intermediate features are linearly superimposed to obtain the multi-spectral mixed solar spectrum simulation data set.

5. The method according to claim 1, characterized in that The step of performing multi-layer convolution on the multi-spectral mixed solar spectrum simulation data set to obtain a solar spectrum image inversion result includes: Based on the multi-spectral mixed solar spectrum simulation data set, a deep convolutional neural network is used to extract semantic information and texture information in the solar spectrum image, and the solar spectrum image inversion result is determined based on the semantic information and the texture information.

6. The method according to claim 5, characterized in that The method further comprises: extracting semantic information and texture information from the solar spectrum image using a deep convolutional neural network based on the multi-spectral mixed solar spectrum simulation data set, and determining the solar spectrum image inversion result based on the semantic information and the texture information. The multi-spectral mixed solar spectrum simulation data set is used as input data, and supervised learning is performed on the output data of the deep convolutional neural network to obtain a trained deep convolutional neural network.

7. A solar spectrum image inversion device based on deep learning, characterized in that: The device comprises: A generating module, used for acquiring a solar spectrum image, and generating a plurality of groups of convolution kernels based on the solar spectrum image by using a point spread function; A fusion module, used for fusing the solar spectrum images containing the target spectrum lines using the multiple groups of convolution kernels to obtain a multi-spectrum mixed solar spectrum simulation data set; The convolution module is used to perform multi-layer convolution on the multi-spectral mixed solar spectrum simulation data set to obtain a solar spectrum image inversion result.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the deep learning-based solar spectrum image inversion method described in any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the solar spectrum image inversion method based on deep learning as described in any one of claims 1 to 6.

10. A computer program product, characterized in that It includes computer instructions, which are used to enable a computer to execute the solar spectrum image inversion method based on deep learning as described in any one of claims 1 to 6.

Images