Response data panchromatic sharpening method and apparatus

Abstract

The present disclosure provides a response data panchromatic sharpening method and device, and relates to the technical field of image processing. First response data with a first spatial resolution and response distribution data with a second spatial resolution are obtained. A data mapping relationship between sub-image blocks of the response distribution data and second response data corresponding to center pixel points of the sub-image blocks is constructed. The data mapping relationship is called, all sub-image blocks of the response distribution data are traversed, and the second response data corresponding to the center pixel points of all the sub-image blocks is output. The second response data corresponding to the center pixel points of all the sub-image blocks is spatially registered with the response distribution data according to the spatial position relationship of each sub-image block in the response distribution data, and third response data with the second spatial resolution is reconstructed. Thus, the acquisition speed of response data (such as hyperspectral data) is improved without sacrificing the image spatial resolution and the image authenticity.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a method and apparatus for full-color sharpening of response data. Background Technology

[0002] Hyperspectral data contains both two-dimensional spatial information and one-dimensional spectral information of the object being measured. It can analyze the two-dimensional distribution of substances and perform spectral analysis on a point in space. It is widely used in fields such as chemical analysis, remote sensing monitoring, food safety, medical diagnosis, and aerospace.

[0003] However, research has shown that hyperspectral data is large in volume and requires long acquisition times. Generally, reducing spatial resolution is used to achieve the acquisition goal, which is not conducive to high-precision image analysis and processing. Summary of the Invention

[0004] The embodiments disclosed herein can improve the acquisition speed of response data (such as hyperspectral data) without sacrificing image spatial resolution and image realism.

[0005] This disclosure provides a method for full-color sharpening of response data in some embodiments, including:

[0006] Acquire a first response data with a first spatial resolution and a response distribution data with a second spatial resolution, the second spatial resolution being greater than the first spatial resolution. The first response data is the response data at a set spatial location, and the response distribution data is the response distribution data of the entire space under a set stress.

[0007] Pair the sub-image blocks of the response distribution data with the one-dimensional second response data in the first response data corresponding to the center pixel of the sub-image block to construct a data mapping relationship between the sub-image blocks of the response distribution data and the second response data corresponding to the center pixel of the sub-image block;

[0008] Invoke the data mapping relationship, traverse all sub-image blocks of the response distribution data, and output the second response data corresponding to the center pixel of each sub-image block;

[0009] Based on the spatial positional relationship of each sub-image block in the response distribution data, the second response data corresponding to the center pixel of all sub-image blocks is spatially registered with the response distribution data, and a third response data with a second spatial resolution is reconstructed.

[0010] In some embodiments, the response distribution data is represented as P∈R W×H×C The sub-image patch of the response distribution data is represented as p∈R w×h×CWhere R represents the real number field, W and H represent the width and height of the response distribution data, respectively, W×H represents the second spatial resolution of the response distribution data, w and h represent the width and height of the sub-image patch, respectively, and C represents the number of channels of the response distribution data and its sub-image patch; the first response data is represented as M∈R N×S The second response data is represented as m∈R 1×S The third response data is represented as D∈R W×H×S Where R represents the real number field, N represents the first spatial resolution of the first response data, S represents the number of channels of the first response data, the second response data and the third response data, and the third response data has a second spatial resolution W×H.

[0011] In some embodiments, pairing sub-image blocks of response distribution data with one-dimensional second response data in the first response data corresponding to the center pixel of the sub-image blocks to construct a data mapping relationship between the sub-image blocks of response distribution data and the second response data corresponding to the center pixel of the sub-image blocks includes:

[0012] Sub-image patches p of the paired response distribution data (i) Input convolutional neural network model Obtain the output of the convolutional neural network model under its current set of model parameters θ.

[0013] Based on the output of the convolutional neural network model under its current parameter set θ and sub-image patch p with response distribution data (i) Paired second response data Based on the gap information between them, the loss function L(θ) is determined;

[0014] Find the set of model parameters θ′ that minimizes the loss function L(θ), and then apply the convolutional neural network model to the set of model parameters θ′. This represents the data mapping relationship between the sub-image blocks of the response distribution data and the second response data corresponding to the center pixel of the sub-image block.

[0015] In some embodiments, the loss function is L(θ) = L0(θ) + λΦ(θ), where, or, Among them, λΦ(θ)=λ||θ||1, or, Where θ represents the set of model parameters that the convolutional neural network model needs to optimize, ||·|| F Let λ represent the F-norm of the matrix, λΦ(θ) represent the regularization term, and λ represent the regularization coefficient.

[0016] In some embodiments, convolutional neural network models include:

[0017] The input layer, Input, is configured to output response distribution data P∈R. W×H×C Cropped to a sub-image patch p of size w×h (i) ∈R w×h×c ;

[0018] The first convolutional layer, Conv1, is configured to crop sub-image patches p. (i) As input, it is convolved with N1 convolution kernels of size w×h, and then processed by an activation function for non-linear mapping, finally outputting N1 feature maps.

[0019] The second convolutional layer, Conv2, is configured to take the output of the first convolutional layer, Conv1. As input, it undergoes convolution with N2 convolution kernels of size w×h, and then passes through an activation function for non-linear mapping and data dimension transformation, ultimately outputting N2 feature values.

[0020] The fully connected layer (FC) is configured to take the output of the second convolutional layer (Conv2) as the input. As input, output S feature values in, All S eigenvalue nodes are with The N2 eigenvalue nodes are connected together.

[0021] In some embodiments, invoking the data mapping relationship, traversing all sub-image blocks of the response distribution data, and outputting the second response data corresponding to the center pixel of all sub-image blocks includes: sequentially inputting all sub-image blocks of the response distribution data into a convolutional neural network model. Obtain the convolutional neural network model The output is the second response data corresponding to the center pixel of each sub-image block.

[0022] In some embodiments, according to the spatial positional relationship of each sub-image block in the response distribution data, the second response data corresponding to the center pixel of all sub-image blocks is spatially registered with the response distribution data, and a third response data with a second spatial resolution is reconstructed, including: placing the second response data corresponding to the center pixel of each sub-image block in the spatial position of each sub-image block in the response distribution data, and reconstructing the third response data with a second spatial resolution from the second response data at each spatial position in the response distribution data.

[0023] In some embodiments, the ratio between the second spatial resolution and the first spatial resolution is greater than 1 and less than 1024.

[0024] In some embodiments, obtaining the first response data includes: sampling a set of one-dimensional second response data at a second spatial resolution for a portion of the location to obtain the first response data, wherein the sampling includes guided sampling or equidistant sampling, wherein the guided sampling includes image segmentation of the sampled navigation map and more dense sampling in the region of interest relative to the background region.

[0025] In some embodiments, acquiring response distribution data includes: selecting one or more characteristic spectral bands of the first response data, extracting their spectral band peak positions and spectral intensities, selecting spectral peak positions based on spectral intensities, and performing single spectral peak imaging at a second spatial resolution at each selected spectral peak position to obtain a single spectral peak image. All single spectral peak images constitute response distribution data at a second spatial resolution.

[0026] In some embodiments, the first response data includes: hyperspectral data, electrical spectrum data, mechanical spectrum data, amplitude spectrum data, phase spectrum data, or frequency spectrum data; the response distribution data corresponding to the first response data includes: a single spectral peak image group, an image group of electrical physical quantities responded by the sample under a given stimulus physical quantity, a force information image group, an amplitude image group, a phase image group, or a frequency image group.

[0027] In some embodiments, response distribution data is acquired; second response data at several points in space is acquired, wherein the second response data at each point in space constitutes first response data; for each second response data point acquired in space, the second response data at that point in space and the corresponding sub-image patch of the response distribution data are paired into a pair of training data to train a convolutional neural network model to construct the data mapping relationship; the convolutional neural network model is invoked to traverse all sub-image patches of the response distribution data and output the second response data corresponding to the center pixel of all sub-image patches; according to the spatial positional relationship of each sub-image patch in the response distribution data, the second response data corresponding to the center pixel of all sub-image patches are spatially registered with the response distribution data, and a third response data with a second spatial resolution is reconstructed; the reconstruction quality is judged, and if the reconstruction quality does not meet the requirements, the second response data at other points in space is acquired and paired into new training data, and the convolutional neural network model is trained again until the reconstruction quality meets the requirements.

[0028] Some embodiments of this disclosure provide a response data full-color sharpening apparatus, including: a memory; and a processor coupled to the memory, the processor being configured to execute a response data full-color sharpening method based on instructions stored in the memory.

[0029] This disclosure provides embodiments of a response data full-color sharpening apparatus, comprising:

[0030] The acquisition module is configured to acquire a first response data with a first spatial resolution and a response distribution data with a second spatial resolution, wherein the second spatial resolution is greater than the first spatial resolution, the first response data is the response data at a set spatial location, and the response distribution data is the response distribution data of the entire space under a set stress.

[0031] The relationship building module is configured to pair sub-image blocks of response distribution data with one-dimensional second response data in the first response data corresponding to the center pixel of the sub-image blocks, and build a data mapping relationship between the sub-image blocks of response distribution data and the second response data corresponding to the center pixel of the sub-image blocks;

[0032] The mapping module is configured to invoke the data mapping relationship, traverse all sub-image blocks of the response distribution data, and output the second response data corresponding to the center pixel of all sub-image blocks.

[0033] The reconstruction module is configured to spatially register the second response data corresponding to the center pixel of each sub-image block with the response distribution data according to the spatial position relationship of each sub-image block in the response distribution data, and reconstruct the third response data with the second spatial resolution.

[0034] In some embodiments, the relationship building module is configured as follows:

[0035] Sub-image patches p of the paired response distribution data (i) Input convolutional neural network model Obtain the output of the convolutional neural network model under its current set of model parameters θ.

[0036] Based on the output of the convolutional neural network model under its current parameter set θ and sub-image patch p with response distribution data (i) Paired second response data Based on the gap information between them, the loss function L(θ) is determined;

[0037] Find the set of model parameters θ′ that minimizes the loss function L(θ), and then apply the convolutional neural network model to the set of model parameters θ′. This represents the data mapping relationship between the sub-image blocks of the response distribution data and the second response data corresponding to the center pixel of the sub-image block.

[0038] Some embodiments of this disclosure propose a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of a response data full-color sharpening method. Attached Figure Description

[0039] The accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. This disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings.

[0040] Obviously, the accompanying drawings described below are merely some embodiments of this disclosure. Those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0041] Figure 1a A flowchart illustrating a response data full-color sharpening method according to some embodiments of this disclosure is shown.

[0042] Figure 1b A flowchart illustrating a hyperspectral data panchromatic sharpening method according to some embodiments of this disclosure is shown.

[0043] Figure 2 The diagram illustrates a process for constructing data mapping relationships according to some embodiments of this disclosure.

[0044] Figure 3 This is a schematic diagram illustrating the output of the second hyperspectral data corresponding to the center pixels of all sub-image blocks in some embodiments of this disclosure.

[0045] Figure 4a The diagram illustrates the functions and processing of four layers in a convolutional neural network model according to some embodiments of this disclosure.

[0046] Figure 4b The illustration shows the effect of recovering third hyperspectral data with second spatial resolution from a first hyperspectral data with first spatial resolution and a single wavenumber image group with second spatial resolution according to some embodiments of the present disclosure.

[0047] Figure 5 This is a schematic diagram of the structure of a response data full-color sharpening device according to some embodiments of this disclosure.

[0048] Figure 6 This is a schematic diagram of the structure of a response data full-color sharpening device according to some embodiments of this disclosure. Detailed Implementation

[0049] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0050] Unless otherwise stated, the terms "first," "second," etc., used in this disclosure are used to distinguish different objects and are not used to indicate size or sequence.

[0051] Figure 1a A flowchart illustrating a response data full-color sharpening method according to some embodiments of this disclosure is shown.

[0052] like Figure 1a As shown, the response data full-color sharpening method of this embodiment includes steps A1-A4.

[0053] Step A1: Obtain a first response data with a first spatial resolution and a response distribution data with a second spatial resolution. The second spatial resolution is greater than the first spatial resolution. The first response data is the response data at a set spatial location, and the response distribution data is the response distribution data of the entire space under the set stress.

[0054] As mentioned earlier, the second spatial resolution is greater than the first spatial resolution; therefore, the ratio between the second spatial resolution and the first spatial resolution is greater than 1. In some embodiments, the ratio between the second spatial resolution and the first spatial resolution is greater than 1 and less than or equal to 1024, such as 2, 512, 1000, etc., but not limited to the examples given. For example, when the ratio between the second spatial resolution and the first spatial resolution is equal to 1000, it means that high-fidelity response data can be recovered using only one-thousandth of the response data and the response distribution data with the second spatial resolution.

[0055] The first response data is, for example, uniform or non-uniform sparse response data.

[0056] In some embodiments, acquiring the first response data includes: sampling a set of one-dimensional second response data at a second spatial resolution for a portion of the location. The "sampling" can be non-uniform sampling or equally spaced sampling along the length or width direction. The "non-uniform sampling" is not random sampling, but guided sampling based on image segmentation. That is, using a faster imaging method, such as nanoscale physicochemical property imaging, a preliminary image of the entire system is created, called a navigation map. This navigation map is analyzed through image segmentation to find important related regions (also called regions of interest). Selectively, denser sampling is applied to the regions of interest (ROIs), while sparser sampling is applied to the background regions.

[0057] With the same number of samples, image reconstruction based on guided sampling is more effective than image reconstruction based on equal-interval sampling.

[0058] In some embodiments, acquiring response distribution data includes: selecting one or more characteristic spectral bands (e.g., spectral bands) of the first response data, extracting their spectral (e.g., spectral) band peak positions and spectral intensities, selecting spectral peak positions based on spectral intensities, and performing single spectral peak imaging at a second spatial resolution at each selected spectral peak position to obtain a single spectral peak position image. All single spectral peak images constitute the response distribution data at the second spatial resolution. Here, a single spectral peak position is also called a single wavenumber, a single spectral peak position image is also called a single wavenumber image, and a single wavenumber image is also called a single panchromatic image. A single panchromatic image generally refers to a high-resolution single-channel image.

[0059] Step A2 involves pairing the sub-image blocks of the response distribution data with the one-dimensional second response data in the first response data corresponding to the center pixel of the sub-image blocks to construct a data mapping relationship between the sub-image blocks of the response distribution data and the second response data corresponding to the center pixel of the sub-image blocks.

[0060] In some embodiments, a convolutional neural network model is constructed, and by training the convolutional neural network model, a data mapping relationship is established between sub-image blocks of response distribution data and second response data corresponding to the center pixels of the sub-image blocks. For example, see the description of steps S21-S27.

[0061] Therefore, by utilizing existing response data and its paired sub-image patches of response distribution data, a data mapping relationship between the sub-image patches of response distribution data and the response data is constructed.

[0062] Step A3: Invoke the data mapping relationship, traverse all sub-image blocks of the response distribution data, and output the second response data corresponding to the center pixel of all sub-image blocks.

[0063] In some embodiments, the trained convolutional neural network model is used to output the second response data corresponding to the center pixels of all sub-image blocks. For example, see the description of step S31.

[0064] Therefore, other response data can be recovered by utilizing the data mapping relationship.

[0065] Step A4: Based on the spatial positional relationship of each sub-image block in the response distribution data, spatially register the second response data corresponding to the center pixel of all sub-image blocks with the response distribution data, and reconstruct the third response data with the second spatial resolution.

[0066] In some embodiments, reconstructing third response data with a second spatial resolution includes: placing the second response data corresponding to the center pixel of each sub-image block at the spatial position of each sub-image block in the response distribution data, and reconstructing the third response data with a second spatial resolution from the second response data at each spatial position in the response distribution data.

[0067] For ease of description, the response distribution data is represented as P∈R. W×H×C The sub-image patch of the response distribution data is represented as p∈R w×h×C Where R represents the real number field, W and H represent the width and height of the response distribution data, respectively, W×H represents the second spatial resolution of the response distribution data, w and h represent the width and height of the sub-image patch, respectively, and C represents the number of channels of the response distribution data and its sub-image patch; the first response data is represented as M∈R N×S The second response data is represented as m∈R 1×S The third response data is represented as D∈R W×H×S , where R represents the real number field, N represents the first spatial resolution of the first response data, S represents the number of channels of the first response data, the second response data and the third response data, and the third response data has a second spatial resolution W×H.

[0068] In the above embodiment, a first response data with a first spatial resolution and a response distribution data with a second spatial resolution (the second spatial resolution being greater than the first spatial resolution) are acquired. A data mapping relationship is constructed between the sub-image blocks of the response distribution data and the one-dimensional second response data corresponding to the center pixels of the sub-image blocks in the first response data by pairing them. This data mapping relationship is then invoked to traverse all sub-image blocks of the response distribution data, outputting the second response data corresponding to the center pixels of all sub-image blocks. Based on the spatial positional relationship of each sub-image block within the response distribution data, spatial registration is performed between the second response data corresponding to the center pixels of all sub-image blocks and the response distribution data, reconstructing a third response data with the second spatial resolution. Thus, the acquisition speed of response data (such as hyperspectral data) is improved without sacrificing image spatial resolution and image realism.

[0069] The response data full-color sharpening method of this disclosure can be applied to online or offline scenarios. In online scenarios, response data can be collected online while the model is trained and the response data is reconstructed. If the reconstruction quality meets the requirements, the collection of response data and model training can be stopped. In offline scenarios, response data for training is acquired, and the model is trained. The model refers to the convolutional neural network model used to construct the data mapping relationship.

[0070] The process of full-color sharpening of response data in online scenarios includes:

[0071] Obtain response distribution data;

[0072] Acquire second response data from several points in space, wherein the second response data from each point in space constitutes the first response data;

[0073] For each second response data point acquired in space, the second response data point in space and the corresponding sub-image patch of response distribution data are paired into a pair of training data to train a convolutional neural network model in order to construct the data mapping relationship.

[0074] The convolutional neural network model is invoked to traverse all sub-image blocks of the response distribution data and output the second response data corresponding to the center pixel of each sub-image block.

[0075] Based on the spatial positional relationship of each sub-image block in the response distribution data, the second response data corresponding to the center pixel of all sub-image blocks is spatially registered with the response distribution data, and a third response data with a second spatial resolution is reconstructed.

[0076] The reconstruction quality is assessed. If the reconstruction quality does not meet the requirements, the second response data of other points in the space is acquired and paired to form new training data. The convolutional neural network model is then trained until the reconstruction quality meets the requirements.

[0077] The embodiments of this disclosure describe response data based on low spatial resolution response data and high spatial resolution response distribution data, which are reconstructed using a panchromatic sharpening method to obtain response data with high spatial resolution. The first response data includes, but is not limited to: hyperspectral data, electrical spectrum data (e.g., current-voltage curves, capacitance-voltage curves, differential capacitance-voltage curves, etc.), mechanical spectrum data (e.g., pressure-voltage curves, pressure-frequency curves, etc.), amplitude spectrum data, phase spectrum data, or frequency spectrum data; the response distribution data corresponding to various first response data includes, but is not limited to: single wavenumber image sets, image sets of electrical physical quantities (e.g., current values) responded by the sample under a given stimulus physical quantity (e.g., a given voltage value), force information image sets, amplitude image sets, phase image sets, or frequency image sets.

[0078] Subsequent embodiments will use the example of reconstructing high spatial resolution hyperspectral data from low spatial resolution hyperspectral data and a high spatial resolution single-wavenumber image set to describe an exemplary hyperspectral data pancolor sharpening method for response data. However, those skilled in the art will understand that other applications of response data and response distribution data are the same as the pancolor sharpening method for this exemplary application of hyperspectral data, differing only in the specific data processed in each application. The data processed in each specific application are listed above.

[0079] Figure 1b A flowchart illustrating a hyperspectral data panchromatic sharpening method according to some embodiments of this disclosure is shown.

[0080] like Figure 1b As shown, the hyperspectral data panchromatic sharpening method of this embodiment includes steps S1-S4.

[0081] Step S1: Acquire a first hyperspectral data with a first spatial resolution and a single wavenumber image group with a second spatial resolution, wherein the second spatial resolution is greater than the first spatial resolution.

[0082] As mentioned earlier, the second spatial resolution is greater than the first spatial resolution; therefore, the ratio between the second spatial resolution and the first spatial resolution is greater than 1. In some embodiments, the ratio between the second spatial resolution and the first spatial resolution is greater than 1 and less than or equal to 1024, such as 2, 512, 1000, etc., but not limited to the examples given. For example, when the ratio between the second spatial resolution and the first spatial resolution is equal to 1000, it means that high-fidelity hyperspectral data can be recovered using only one-thousandth of the hyperspectral data and a single-wavenumber image group with the second spatial resolution.

[0083] The first type of hyperspectral data is, for example, uniform or non-uniform sparse hyperspectral data.

[0084] In some embodiments, acquiring the first hyperspectral data includes: sampling a set of one-dimensional second hyperspectral data at a second spatial resolution for a portion of the location. The "sampling" can be non-uniform sampling or equally spaced sampling along the length or width direction. The "non-uniform sampling" is not random sampling, but guided sampling based on image segmentation. That is, using a faster imaging method, such as nanoscale physicochemical property imaging, a preliminary image of the entire system is created, called a navigation map, to identify important related regions (also called regions of interest). This navigation map is then analyzed through image segmentation, selectively setting denser sampling in the regions of interest (ROIs) and sparser sampling in the background regions. With the same number of samples, image reconstruction based on guided sampling is more effective than image reconstruction based on equally spaced sampling.

[0085] In some embodiments, acquiring a single-wavenumber image group includes: selecting spectral peak positions (possibly one or more spectral peak positions) of a first hyperspectral data set, performing high-resolution (second spatial resolution) single-wavenumber imaging at each spectral peak position to obtain a single-wavenumber image, and forming a single-wavenumber image group with the second spatial resolution from all single-wavenumber images. A single-wavenumber image is also called a single panchromatic image, which generally refers to a high-resolution single-channel image.

[0086] Step S2 involves pairing the sub-image blocks of a single wavenumber image group with the one-dimensional second hyperspectral data in the first hyperspectral data corresponding to the center pixel of the sub-image block, thereby constructing a data mapping relationship between the sub-image blocks of the single wavenumber image group and the second hyperspectral data corresponding to the center pixel of the sub-image block.

[0087] In some embodiments, a convolutional neural network model is constructed, and by training the convolutional neural network model, a data mapping relationship is established between sub-image blocks of a single wavenumber image group and the second hyperspectral data corresponding to the center pixel of the sub-image block. See the description of steps S21-S27 for details.

[0088] Therefore, by utilizing existing hyperspectral data and its paired sub-image patches of single-wavenumber image groups, a data mapping relationship between the sub-image patches of single-wavenumber image groups and hyperspectral data can be constructed.

[0089] Step S3: Invoke the data mapping relationship, traverse all sub-image blocks of the single wavenumber image group, and output the second hyperspectral data corresponding to the center pixel of all sub-image blocks.

[0090] In some embodiments, the trained convolutional neural network model is used to output the second hyperspectral data corresponding to the center pixels of all sub-image blocks. See the description of step S31 for details.

[0091] Therefore, other hyperspectral data can be recovered by utilizing data mapping relationships.

[0092] Step S4: According to the spatial position relationship of each sub-image block in the single wavenumber image group, the second hyperspectral data corresponding to the center pixel of all sub-image blocks are spatially registered with the single wavenumber image group, and the third hyperspectral data with the second spatial resolution is reconstructed.

[0093] In some embodiments, reconstructing third hyperspectral data with a second spatial resolution includes: placing the second hyperspectral data corresponding to the center pixel of each sub-image block at the spatial position of each sub-image block in a single wavenumber image group, and reconstructing the third hyperspectral data with a second spatial resolution from the second hyperspectral data at each spatial position in the single wavenumber image group.

[0094] The above embodiment acquires a first hyperspectral data set with a first spatial resolution and a single-wavenumber image group with a second spatial resolution greater than the first spatial resolution. By pairing sub-image blocks of the single-wavenumber image group with the one-dimensional second hyperspectral data corresponding to the center pixels of the sub-image blocks, a data mapping relationship is constructed between the sub-image blocks and the second hyperspectral data corresponding to the center pixels of the sub-image blocks. This data mapping relationship is then invoked to traverse all sub-image blocks of the single-wavenumber image group, outputting the second hyperspectral data corresponding to the center pixels of all sub-image blocks. Based on the spatial position of each sub-image block within the single-wavenumber image group, spatial registration is performed between the second hyperspectral data corresponding to the center pixels of all sub-image blocks and the single-wavenumber image group, reconstructing a third hyperspectral data set with the second spatial resolution. Thus, the acquisition speed of hyperspectral data is improved without sacrificing image spatial resolution and image fidelity.

[0095] For ease of description, the data formats of single wavenumber image groups and various hyperspectral data are represented as follows.

[0096] A single wavenumber image group is represented as P∈R W×H×C The sub-image patch of a single wavenumber image group is represented as p∈R w×h×C Where R represents the real number domain, W and H represent the width and height of a single wavenumber image group, respectively, W×H represents the second spatial resolution of the single wavenumber image group, w and h represent the width and height of a sub-image patch, respectively, and C represents the number of channels of the single wavenumber image group and its sub-image patches. The single wavenumber image group and its sub-image patches have the same number of channels.

[0097] The first hyperspectral data is represented as M∈R N×S The second hyperspectral data is represented as m∈R 1×S The third hyperspectral data is represented as D∈R W×H×S Where R represents the real number domain, N represents the first spatial resolution of the first hyperspectral data, and S represents the number of spectral channels of the first, second, and third hyperspectral data. That is, the first, second, and third hyperspectral data have the same number of spectral channels, and the third hyperspectral data has a second spatial resolution W×H.

[0098] As can be seen, the sub-image patch is three-dimensional, while the second hyperspectral data is one-dimensional. The aforementioned data mapping relationship realizes the mapping from three-dimensional data to one-dimensional data.

[0099] Figure 2 The diagram illustrates a process for constructing data mapping relationships according to some embodiments of this disclosure.

[0100] like Figure 2 As shown, the construction of the data mapping relationship in this embodiment includes steps S21-S27, wherein steps S21-24 involve building a convolutional neural network model, and steps S25-27 involve training the convolutional neural network model, thereby finally constructing a data mapping relationship between the sub-image blocks of a single wavenumber image group and the second hyperspectral data corresponding to the center pixel of the sub-image block.

[0101] Step S21, the input layer is configured to input a single wavenumber image group P∈R W×H×C Cropped to a sub-image patch p of size w×h (i) ∈R w×h×C .

[0102] In step S22, the first convolutional layer Conv1 is configured to crop sub-image blocks p (i) As input, it is convolved with N1 convolution kernels of size w×h, and then processed by an activation function for non-linear mapping, finally outputting N1 feature maps.

[0103] The convolution process can be applied to p (i) No boundary padding is performed. For example, a 10x10 image, after being convolved with a 3x3 kernel, will become an 8x8 image. The convolution process can also be applied to p... (i) Perform boundary padding. For example, the zero-padding method first pads a 10x10 image with a ring of zeros, turning it into a 12x12 image, and then convolves it with a 3x3 kernel, which will turn it back into a 10x10 image. Alternatively, the boundary padding method does not pad with zeros, but instead mirrors and pads the image outward based on relevant edge data.

[0104] The activation function can be ReLU, or it can be Sigmoid, tanh, maxout, etc., but is not limited to the examples given.

[0105] In step S23, the second convolutional layer Conv2 is configured to use the output of the first convolutional layer Conv1. As input, it undergoes convolution with N2 convolution kernels of size w×h, and then passes through an activation function for non-linear mapping and data dimension transformation, ultimately outputting N2 feature values.

[0106] Incorrect convolution process Perform boundary filling.

[0107] Step S24, the fully connected layer FC is configured to output the second convolutional layer Conv2. As input, output S feature values in, All S eigenvalue nodes are with The N2 eigenvalue nodes are connected together.

[0108] Step S25, the sub-image blocks p of the paired single-wavenumber image groups (i) Input convolutional neural network model Obtain the output of the convolutional neural network model under its current set of model parameters θ.

[0109] Convolutional Neural Network Model That is, the convolutional neural network model built through steps S21-S24.

[0110] Step S26, based on the output of the convolutional neural network model under its current parameter set θ and sub-image blocks p with single wavenumber image groups (i) Paired second hyperspectral data The difference information between them is used to determine the loss function L(θ).

[0111] The loss function is, for example, L(θ) = L0(θ) + λΦ(θ).

[0112] in, or,

[0113] Among them, λΦ(θ)=λ||θ||1, or,

[0114] Where θ represents the set of model parameters that the convolutional neural network model needs to optimize, ||·|| FLet λ represent the F-norm of the matrix, λΦ(θ) represent the regularization term, which is the penalty term in the loss function and can impose some restrictions on the parameter θ in the loss function. λ represents the regularization coefficient. The meanings of other symbols are as described above and will not be repeated here.

[0115] Step S27: Using, for example, a gradient descent algorithm, find the model parameter set θ′ that minimizes the loss function L(θ), and then convert the convolutional neural network model under the model parameter set θ′. This refers to the data mapping relationship between sub-image blocks of a single wavenumber image group and the second hyperspectral data corresponding to the center pixel of the sub-image block.

[0116] In the above embodiments, by building and training a convolutional neural network model, a data mapping relationship is ultimately constructed between the sub-image blocks of a single wavenumber image group and the second hyperspectral data corresponding to the center pixel of the sub-image block.

[0117] Figure 3 This is a schematic diagram illustrating the output of the second hyperspectral data corresponding to the center pixels of all sub-image blocks in some embodiments of this disclosure.

[0118] like Figure 3 As shown, outputting the second hyperspectral data corresponding to the center pixels of all sub-image blocks includes: step S31.

[0119] Step S31: Fill the boundaries of the single-wavenumber image group P according to the width w and height h of the sub-image patches of the single-wavenumber image group; input all sub-image patches of the single-wavenumber image group into the trained convolutional neural network model in sequence. Obtain the convolutional neural network model The output is the second hyperspectral data corresponding to the center pixel of each sub-image block.

[0120]

[0121] In the above embodiment, the trained convolutional neural network model is used to output the second hyperspectral data corresponding to the center pixels of all sub-image blocks.

[0122] To make the solutions disclosed herein clearer, some application examples are provided below to aid in understanding the solutions disclosed herein. It should be understood that the specific parameters used in the application examples are merely illustrative and should not be considered as limitations. Furthermore, for the same steps, the application examples use the same notation as the foregoing embodiments to make the logical correspondence between the application examples and the foregoing embodiments clearer.

[0123] Step S1: Obtain first hyperspectral data m∈R with a first spatial resolution. 137×512 A single-wavenumber image group P∈R with second spatial resolution360×180×3 The spatial resolution of the hyperspectral data is N = 137, and the spatial resolution of the single wavenumber image group is 64800 = (360 × 180) = W × H, which satisfies the range 1 < (W × H) / N < 1024.

[0124] Step S2: Pair the sub-image blocks of the single wavenumber image group with the second hyperspectral data in the first hyperspectral data corresponding to the center pixel of the sub-image block, so that the sub-image block p∈R of the single wavenumber image group is paired. 3×3×3 (3D data) is used as input for data mapping, with the second hyperspectral data m∈R corresponding to the center pixel of the sub-image patch. 1×512 (1D data) is used as the output of the data mapping to calculate and analyze the data mapping relationship between the two from 3D to 1D.

[0125] Step S21, Input layer, crop a 3×3 sub-image patch p from the single wavenumber image group. (i) ∈R 3 ×3×3 .

[0126] Step S22, the first convolutional layer Conv1, with the cropped sub-image patch p (i) As input, it is convolved with 64 convolution kernels of size 3×3. The convolution process affects p. (i) Boundary padding is performed, followed by non-linear mapping using an activation function, ultimately outputting 64 feature maps.

[0127] Step S23, the second convolutional layer Conv2 takes the output of the first convolutional layer Conv1 as input. As input, it undergoes convolution operations with 256 convolution kernels of size 3×3. The convolution process is not... Boundary padding is performed, followed by non-linear mapping and data dimensionality transformation using an activation function, ultimately outputting 256 feature values, i.e.

[0128] Step S24, fully connected layer FC, with the output of the second convolutional layer Conv2. As input, the output consists of 512 feature values, i.e. in, All 512 eigenvalue nodes are related to The 256 eigenvalue nodes are connected.

[0129] Figure 4a The diagram illustrates the functions and processing of four layers in a convolutional neural network model according to some embodiments of this disclosure.

[0130] Step S25: Extract sub-image blocks p from the paired single-wavenumber image groups.(i) Input the convolutional neural network model defined in steps S21 to S24, and obtain the output of the convolutional neural network model under its current model parameter set θ.

[0131] Step S26, based on the output of the convolutional neural network model under its current parameter set θ and sub-image blocks p with single wavenumber image groups (i) Paired second hyperspectral data The difference information between them is defined by the loss function as L(θ) = L0(θ) + λΦ(θ).

[0132] in, or,

[0133] Among them, λΦ(θ)=λ||θ||1, or,

[0134] Where θ represents the set of model parameters that the convolutional neural network model needs to optimize, ||·|| F Let λ represent the F-norm of the matrix, λΦ(θ) represent the regularization term, which is the penalty term in the loss function and can impose some restrictions on the parameter θ in the loss function. λ represents the regularization coefficient. The meanings of other symbols are as described above and will not be repeated here.

[0135] Step S27: Using, for example, a gradient descent algorithm, find the model parameter set θ′ that minimizes the loss function L(θ), and then convert the convolutional neural network model under the model parameter set θ′. This refers to the data mapping relationship between sub-image blocks of a single wavenumber image group and the second hyperspectral data corresponding to the center pixel of the sub-image block.

[0136] Step S3: Call the data mapping relationship, traverse all sub-image blocks of the single wavenumber image group, and output the second hyperspectral data corresponding to the center pixel of all sub-image blocks.

[0137] Step S31: Based on the width and height (3×3) of the sub-image blocks of the single-wavenumber image group P, perform boundary padding on the single-wavenumber image group P; then sequentially input all sub-image blocks of the single-wavenumber image group into the trained convolutional neural network model. Obtain the convolutional neural network model The output is the second hyperspectral data corresponding to the center pixel of each sub-image block.

[0138]

[0139] Step S4: According to the spatial relationship, the second hyperspectral data obtained by using the data mapping relationship is... Spatial registration was performed with the single-wavenumber image group P to obtain the second hyperspectral data. Reconstruct hyperspectral data D∈R with second spatial resolution 360×180×512 .

[0140] In the above application example, by acquiring a first hyperspectral data set with a first spatial resolution and a single-wavenumber image group with a second spatial resolution (the second spatial resolution being greater than the first), a data mapping relationship is constructed between the sub-image blocks of the single-wavenumber image group and the one-dimensional second hyperspectral data corresponding to the center pixels of the first hyperspectral data set. This data mapping relationship is then invoked to traverse all sub-image blocks of the single-wavenumber image group, outputting the second hyperspectral data corresponding to the center pixels of all sub-image blocks. Based on the spatial position of each sub-image block within the single-wavenumber image group, spatial registration is performed between the second hyperspectral data corresponding to the center pixels of all sub-image blocks and the single-wavenumber image group, reconstructing a third hyperspectral data set with the second spatial resolution. Thus, the acquisition speed of hyperspectral data is improved without sacrificing image spatial resolution and image fidelity.

[0141] A schematic diagram illustrating the effect of recovering a third hyperspectral data with a second spatial resolution from a first hyperspectral data set with a first spatial resolution and a single wavenumber image set with a second spatial resolution is shown below. Figure 4b As shown.

[0142] Figure 5 This is a schematic diagram of the structure of a response data full-color sharpening device according to some embodiments of this disclosure.

[0143] like Figure 5 As shown, the response data full-color sharpening device 500 of this embodiment includes modules 510-540.

[0144] The acquisition module 510 is configured to acquire a first response data with a first spatial resolution and a response distribution data with a second spatial resolution greater than the first spatial resolution.

[0145] The relationship construction module 520 is configured to construct a data mapping relationship between the sub-image blocks of the response distribution data and the second response data corresponding to the center pixel of the sub-image blocks by pairing the sub-image blocks of the response distribution data with the one-dimensional second response data in the first response data corresponding to the center pixel of the sub-image blocks.

[0146] The mapping module 530 is configured to invoke the data mapping relationship, traverse all sub-image blocks of the response distribution data, and output the second response data corresponding to the center pixel of all sub-image blocks.

[0147] The reconstruction module 540 is configured to spatially register the second response data corresponding to the center pixel of each sub-image block with the response distribution data according to the spatial position relationship of each sub-image block in the response distribution data, and reconstruct the third response data with the second spatial resolution.

[0148] In some embodiments, when acquiring the first response data, the acquisition module 510 is configured to sample a set of one-dimensional second response data at a second spatial resolution to obtain the first response data. The sampling includes guided sampling or equidistant sampling. The guided sampling includes image segmentation of the sampled navigation map and more dense sampling in the region of interest relative to the background region.

[0149] In some embodiments, when acquiring response distribution data, the acquisition module 510 is configured to select one or more characteristic spectral bands of the first response data, extract the spectral band peak positions and spectral intensities, select spectral peak positions according to the spectral intensities, and perform single-wavenumber imaging at a second spatial resolution at each selected spectral peak position to obtain a single-wavenumber image. All single-wavenumber images constitute response distribution data at a second spatial resolution.

[0150] In some embodiments, the relationship building module 520 is configured as follows:

[0151] Sub-image patches p of the paired response distribution data (i) Input convolutional neural network model Obtain the output of the convolutional neural network model under its current set of model parameters θ.

[0152] Based on the output of the convolutional neural network model under its current parameter set θ and sub-image patch p with response distribution data (i) Paired second response data Based on the gap information between them, the loss function L(θ) is determined;

[0153] Using algorithms such as gradient descent, we find the set of model parameters θ′ that minimizes the loss function L(θ), and then apply this parameter to the convolutional neural network model. This represents the data mapping relationship between the sub-image blocks of the response distribution data and the second response data corresponding to the center pixel of the sub-image block.

[0154] In some embodiments, the mapping module 530 is configured to sequentially input all sub-image patches of the response distribution data into a convolutional neural network model. Obtain the convolutional neural network model The output is the second response data corresponding to the center pixel of each sub-image block.

[0155] In some embodiments, the reconstruction module 540 is configured to place the second response data corresponding to the center pixel of each sub-image block at the spatial position of each sub-image block in the response distribution data, and reconstruct the second response data at each spatial position in the response distribution data to generate third response data with a second spatial resolution.

[0156] Figure 6 This is a schematic diagram of the structure of a response data full-color sharpening device according to some embodiments of this disclosure.

[0157] like Figure 6 As shown, the response data full-color sharpening apparatus 600 of this embodiment includes: a memory 610 and a processor 620 coupled to the memory 610. The processor 620 is configured to execute the response data full-color sharpening method of any of the foregoing embodiments based on instructions stored in the memory 610.

[0158] The memory 610 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory may store, for example, the operating system, application programs, boot loader, and other programs.

[0159] The processor 620 can be implemented using discrete hardware components such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistors.

[0160] The device 600 may also include an input / output interface 630, a network interface 640, and a storage interface 650. These interfaces 630, 640, and 650, as well as the memory 610 and processor 620, can be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a monitor, mouse, keyboard, and touchscreen. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0161] Some embodiments of this disclosure provide a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the response data full-color sharpening method of various embodiments.

[0162] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more non-transitory computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer program code.

[0163] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0164] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0165] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0166] The above description is only a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A method for panchromatic sharpening of response data, characterized in that, include: Acquire a first response data with a first spatial resolution and a response distribution data with a second spatial resolution, the second spatial resolution being greater than the first spatial resolution. The first response data is the response data at a set spatial location, and the response distribution data is the response distribution data of the entire space under a set stress. Pairing sub-image blocks of response distribution data with one-dimensional second response data in the first response data corresponding to the center pixel of the sub-image blocks, constructing a data mapping relationship between the sub-image blocks of response distribution data and the second response data corresponding to the center pixel of the sub-image blocks, including: pairing sub-image blocks p of the response distribution data to be paired (i) Input convolutional neural network model Obtain the output of the convolutional neural network model under its current set of model parameters θ. Based on the output of the convolutional neural network model under its current model parameter set θ and sub-image patch p with response distribution data (i) Paired second response data The gap information between them is used to determine the loss function L(θ); the set of model parameters θ that minimizes the loss function L(θ) is found. ′ , the model parameter set θ ′ Convolutional Neural Network Model The data mapping relationship between the sub-image block of the response distribution data and the second response data corresponding to the center pixel of the sub-image block; Invoke the data mapping relationship, traverse all sub-image blocks of the response distribution data, and output the second response data corresponding to the center pixel of each sub-image block; Based on the spatial positional relationship of each sub-image block in the response distribution data, the second response data corresponding to the center pixel of all sub-image blocks is spatially registered with the response distribution data, and a third response data with a second spatial resolution is reconstructed.

2. The method according to claim 1, characterized in that, The response distribution data is represented as P∈R W×H×C The sub-image patch of the response distribution data is represented as p∈R w×h×C Where R represents the real number field, W and H represent the width and height of the response distribution data, respectively, W×H represents the second spatial resolution of the response distribution data, w and h represent the width and height of the sub-image patch, respectively, and C represents the number of channels of the response distribution data and its sub-image patch; The first response data is represented as M∈R N×S The second response data is represented as m∈R 1×S The third response data is represented as D∈R W ×H×S Where R represents the real number field, N represents the first spatial resolution of the first response data, S represents the number of channels of the first response data, the second response data and the third response data, and the third response data has a second spatial resolution W×H.

3. The method according to claim 1, characterized in that, The loss function is L(θ) = L0(θ) + λΦ(θ). in, or, Where, λΦ(θ)=λ‖θ‖1, or, Where θ represents the set of model parameters that the convolutional neural network model needs to optimize, ‖·‖ F Let λ represent the F-norm of the matrix, λΦ(θ) represent the regularization term, and λ represent the regularization coefficient.

4. The method according to claim 1, characterized in that, Convolutional Neural Network Model include: The input layer, Input, is configured to output response distribution data P∈R. W×H×C Cropped to a sub-image patch p of size w×h (i) ∈R w ×h×C ; The first convolutional layer, Conv1, is configured to crop sub-image patches p. (i) As input, it is convolved with N1 convolution kernels of size w×h, and then processed by an activation function for non-linear mapping, finally outputting N1 feature maps. The second convolutional layer, Conv2, is configured to take the output of the first convolutional layer, Conv1. As input, it undergoes convolution with N2 convolution kernels of size w×h, and then passes through an activation function for non-linear mapping and data dimension transformation, ultimately outputting N2 feature values. The fully connected layer (FC) is configured to take the output of the second convolutional layer (Conv2) as the input. As input, output S feature values in, All S eigenvalue nodes are with The N2 eigenvalue nodes are connected together.

5. The method according to claim 1, characterized in that, The data mapping relationship is invoked, and all sub-image blocks of the response distribution data are traversed. The second response data corresponding to the center pixel of each sub-image block is output, including: All sub-image patches of the response distribution data are sequentially input into the convolutional neural network model. Obtain the convolutional neural network model The output is the second response data corresponding to the center pixel of each sub-image block.

6. The method according to claim 1, characterized in that, Based on the spatial positional relationship of each sub-image block in the response distribution data, the second response data corresponding to the center pixel of all sub-image blocks is spatially registered with the response distribution data, and a third response data with a second spatial resolution is reconstructed, including: The second response data corresponding to the center pixel of each sub-image block is placed at the spatial position of each sub-image block in the response distribution data, and the second response data at each spatial position in the response distribution data is reconstructed to generate third response data with a second spatial resolution.

7. The method according to any one of claims 1-6, characterized in that, The ratio between the second spatial resolution and the first spatial resolution is greater than 1 and less than 1024.

8. The method according to any one of claims 1-6, characterized in that, Obtaining first response data includes: The first response data is obtained by sampling a set of one-dimensional second response data at a second spatial resolution for a portion of the location. The sampling includes guided sampling or equidistant sampling. The guided sampling includes image segmentation of the sampled navigation map and more dense sampling in the region of interest relative to the background region.

9. The method according to any one of claims 1-6, characterized in that, Obtaining response distribution data includes: One or more characteristic spectral bands of the first response data are selected, and their peak positions and spectral intensities are extracted. The peak positions are selected based on the spectral intensities. At each selected peak position, a single peak position image with second spatial resolution is obtained. All single peak position images constitute the response distribution data with second spatial resolution.

10. The method according to any one of claims 1-6, characterized in that, The first response data includes: hyperspectral data, electrical spectrum data, mechanical spectrum data, amplitude spectrum data, phase spectrum data, or frequency spectrum data; The response distribution data corresponding to the first response data include: a single spectral peak image group, an image group of the electrical physical quantities responded by the sample under a given stimulus physical quantity, a force information image group, an amplitude image group, a phase image group, or a frequency image group.

11. The method according to claim 1, characterized in that, include: Obtain response distribution data; Acquire second response data from several points in space, wherein the second response data from each point in space constitutes the first response data; For each second response data point acquired in space, the second response data point in space and the corresponding sub-image patch of response distribution data are paired into a pair of training data to train a convolutional neural network model in order to construct the data mapping relationship. The convolutional neural network model is invoked to traverse all sub-image blocks of the response distribution data and output the second response data corresponding to the center pixel of each sub-image block. Based on the spatial positional relationship of each sub-image block in the response distribution data, the second response data corresponding to the center pixel of all sub-image blocks is spatially registered with the response distribution data, and a third response data with a second spatial resolution is reconstructed. The reconstruction quality is assessed. If the reconstruction quality does not meet the requirements, the second response data of other points in the space is acquired and paired to form new training data. The convolutional neural network model is then trained until the reconstruction quality meets the requirements.

12. A response data full-color sharpening device, comprising: Memory; as well as A processor coupled to the memory, the processor being configured to execute the response data full-color sharpening method according to any one of claims 1-11 based on instructions stored in the memory.

13. A response data full-color sharpening device, comprising: The acquisition module is configured to acquire a first response data with a first spatial resolution and a response distribution data with a second spatial resolution, wherein the second spatial resolution is greater than the first spatial resolution, the first response data is the response data at a set spatial location, and the response distribution data is the response distribution data of the entire space under a set stress. The relationship construction module is configured to pair sub-image blocks of response distribution data with one-dimensional second response data in the first response data corresponding to the center pixel of the sub-image blocks, and construct a data mapping relationship between the sub-image blocks of response distribution data and the second response data corresponding to the center pixel of the sub-image blocks, including: pairing sub-image blocks p of the response distribution data. (i) Input convolutional neural network model Obtain the output of the convolutional neural network model under its current set of model parameters θ. Based on the output of the convolutional neural network model under its current model parameter set θ and sub-image patch p with response distribution data (i) Paired second response data The gap information between them is used to determine the loss function L(θ); the set of model parameters θ that minimizes the loss function L(θ) is found. ′ , the model parameter set θ ′ Convolutional Neural Network Model The data mapping relationship between the sub-image block of the response distribution data and the second response data corresponding to the center pixel of the sub-image block; The mapping module is configured to invoke the data mapping relationship, traverse all sub-image blocks of the response distribution data, and output the second response data corresponding to the center pixel of all sub-image blocks. The reconstruction module is configured to spatially register the second response data corresponding to the center pixel of each sub-image block with the response distribution data according to the spatial position relationship of each sub-image block in the response distribution data, and reconstruct the third response data with the second spatial resolution.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the response data full-color sharpening method according to any one of claims 1-11.

Images