A weight training method for multi-point hyperspectral binary classification

Abstract

The application discloses a weight training method for multi-point hyperspectral binary classification. The method adopts a device based on a composite medium grating structure, which can sequentially complete linear response and harmonic nonlinear operation of light. The specific method comprises the following steps: (1) establishing a preliminary model according to the linear response and harmonic nonlinear operation completed by the device; (2) considering the actual limitation of the device, adding a dual operation to the preliminary model according to the requirement of the classification task, and applying a constraint condition to obtain a classification model; (3) using an open-source hyperspectral classification data set, adding noise to the training data and interpolating, performing binary processing on the label, and finally using a gradient descent method to update the weight of the classification model until convergence. The application effectively solves the weight training problem of multi-point hyperspectral binary classification for the device based on the composite medium grating structure, and can realize the on-chip hyperspectral multi-point classification task.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image pattern recognition, and particularly relates to a weight training method for multi-point hyperspectral binary classification. BACKGROUND

[0002] Hyperspectral image processing is a basic problem in the field of image processing. In recent years, using hyperspectral remote sensing to observe the earth has received more and more attention and has been widely used in target detection and identification, classification, monitoring and other fields. Since hyperspectral remote sensing can obtain information of multiple spectral bands, it has stronger ability to distinguish objects than traditional RGB images, so using hyperspectral images to complete the classification task has certain advantages.

[0003] The device based on the composite medium grating structure is a new generation of imaging device. This device can realize the nonlinear calculation of the harmonic mean of electrical signals and optical signals by using its intrinsic physical characteristics, so it is very suitable for storing weight electrical signals and receiving hyperspectral optical signals, thereby it can conveniently complete the task of multi-point hyperspectral classification. However, at present, there is no weight training method for multi-point hyperspectral binary classification for this kind of device.

[0004] With the development of machine learning and deep learning, a series of classification algorithms such as supervised learning, semi-supervised learning and unsupervised learning have appeared. However, due to the actual limitations of the device based on the composite medium grating structure, these unconstrained training methods cannot be directly applied to this kind of device. SUMMARY

[0005] In view of the defects in the above technology, the purpose of the present application is to provide a weight training method for multi-point hyperspectral binary classification, which is especially suitable for this kind of device based on the composite medium grating structure.

[0006] The technical scheme adopted by the method of the present application is as follows:

[0007] A weight training method for multi-point hyperspectral binary classification adopts a device based on a composite medium grating structure, which can sequentially complete linear response and harmonic nonlinear operation of light. The weight training method comprises the following steps:

[0008] (1) establishing a preliminary model according to the linear response and harmonic nonlinear operation completed by the device;

[0009] (2) considering the actual limitations of the device, adding a dual operation to the preliminary model combined with the requirements of the classification task, and applying a constraint condition to obtain a classification model;

[0010] (3) using an open source hyperspectral classification dataset, adding noise to the training data and interpolating, binarizing the labels, and finally using gradient descent to update the weights of the classification model until convergence.

[0011] Further, in the step (1), the preliminary model is:

[0012]

[0013] wherein M is the number of bands of the hyperspectral data, w i is the i-th value of the weight parameter w, y=kx+b is the linear response of the device to a single pixel point x of the hyperspectral image, y i is the i-th band value of the linear response y, y out is the output of the single pixel point of the hyperspectral image after passing through the device.

[0014] Further, in the step (2), the actual restriction of the device is specifically that only positive weights can be stored.

[0015] Further, the addition of dual operations is specifically that two sets of weight values w1 and w2 are used to calculate the linear response y respectively, and the obtained values are subtracted:

[0016]

[0017] Further, the constraint condition imposed on the preliminary model is specifically that the weight is constrained to be positive, that is:

[0018]

[0019] Further, the device based on the composite dielectric gate structure comprises a composite dielectric gate photosensitive detector and a composite dielectric gate transistor formed above the same P-type semiconductor substrate; the composite dielectric gate photosensitive detector is provided with a first bottom insulating dielectric layer, a first floating gate, a first top insulating dielectric layer and a first control gate above the substrate in sequence, and is provided with a source and a drain on both sides in the substrate; the composite dielectric gate transistor is provided with a second bottom insulating dielectric layer, a second floating gate, a second top insulating dielectric layer and a second control gate above the substrate in sequence, and is provided with a source and a drain on both sides in the substrate; the source of the composite dielectric gate photosensitive detector is shared with the source of the composite dielectric gate transistor.

[0020] Further, in the step (3), the noise added to the training data is specifically Gaussian noise, impulse noise and strip noise.

[0021] Further, after the step (3), the test set of the open source dataset is used to test the classification model, and the precision and recall of the classification are obtained.

[0022] The method of the present application is based on considering the actual limitations of such devices based on composite dielectric gate structures, first simulating the algorithm and applying constraints, and then training and testing the model to obtain the weight value using the neural network framework and open source dataset. The present application effectively solves the weight training problem for such devices based on composite dielectric gate structures to complete multi-point hyperspectral two-classification, and can realize the task of on-chip hyperspectral multi-point classification. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 A device structure based on a composite dielectric gate structure in an embodiment of the present application.

[0024] Figure 2 A weight training method flowchart in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0026] The method of the present application is directed to a device based on a composite dielectric gate structure, which can sequentially complete linear response to light and harmonic nonlinear operation. An exemplary structure of a device based on a composite dielectric gate structure is shown in Figure 1 , which includes a composite dielectric gate photosensitive detector and a composite dielectric gate transistor formed above the same P-type semiconductor substrate. Among them, the composite dielectric gate photosensitive detector is provided with a first bottom insulating dielectric layer, a first floating gate, a first top insulating dielectric layer, and a first control gate above the substrate in sequence, and is provided with a source and a drain on both sides in the substrate; the composite dielectric gate transistor is provided with a second bottom insulating dielectric layer, a second floating gate, a second top insulating dielectric layer, and a second control gate above the substrate in sequence, and is provided with a source and a drain on both sides in the substrate. Among them, the source of the composite dielectric gate photosensitive detector is shared with the source of the composite dielectric gate transistor. The specific structure and working process of the device can be referred to CN114841847A.

[0027] The present embodiment provides a weight training method for the above-mentioned device based on a composite dielectric gate structure to complete multi-point hyperspectral two-classification, and the specific steps are shown in Figure 2 .

[0028] Step 1, since the above-mentioned device can sequentially complete linear response to light and harmonic nonlinear operation, a preliminary algorithm model is established according to the operation mode. Specifically, for input x, the response of the device to it is approximately linear, i.e. output y=kx+b, where k and b are constants, in this example k=1, b=1; for harmonic operation, the device can store weight value w, and perform harmonic operation and summation on the weight value and input, i.e. for each pixel point, the output of the device can be represented by the following formula

[0029]

[0030] where M is the number of bands of hyperspectral data, w i is the i-th value of weight parameter w, y i = kx i +b is the i-th band value.

[0031] Step 2, impose positive value constraint on the algorithm model according to the actual limitation of the device, and add dual operation combined with the requirements of the classification task. As shown in Figure 1 , since the number of stored electrons of the device based on composite dielectric gate structure is positive, only positive weights can be stored, so it is necessary to impose positive value constraint on the weights of the model. At the same time, since the classification result is usually taken as 0 in the classification task, if the output of the model is all positive, it will lead to difficulty in determining the classification threshold, therefore, in order to ensure the accuracy and threshold invariability of the classification task, the same dual operation as formula (1) can be added, that is, using two sets of weight values w1 and w2 to calculate the input y respectively, and subtracting the values obtained

[0032]

[0033] After adding the dual operation, the model can classify the results with 0 as the threshold. When optimizing the model, the result of formula (2) and the label can be compared to optimize the weight, but there is no constraint on the search space of the weight. Based on the actual situation, it is necessary to impose positive value constraint on the weight w ai ,a∈{1,2}, therefore a relatively direct method is to impose absolute value constraint on the weight, that is,

[0034]

[0035] Step 3, use the open source hyperspectral classification data set, add noise to the training data and interpolation, binarize the label, and finally use gradient descent method to update the parameters of the model until convergence. The data set uses Indian_pines taken by AVIRIS spectrometer, which has a total of 145x145 pixel points, each point has 220 bands. 80% of the pixel points are selected as the training set, and 20% of the pixel points are selected as the test set.

[0036] Since real hyperspectral data is often affected by environmental noise, in order to simulate the actual situation, the data set needs to be preprocessed first. Add Gaussian noise with mean 0 and standard deviation 5% of the maximum pixel value to each band; add impulse noise to each band, with the number of noise points being 20% of the randomly selected pixels, of which 50% are black points and the rest are white points; randomly select 5% of the bands to add 3 strip noises. At the same time, since the label of the open source data set usually has many categories, but this task only needs to distinguish between foreground and background, so the positive example with the most pixel value in the data set is taken as the only positive example, and the rest are regarded as negative examples.

[0037] Specifically, in training the weights, the PyTorch neural network framework is used for training. For each pixel y, first randomly initialize the weight values w1 and w2, perform forward propagation to get the output y out , and use binary classification cross-entropy loss as the optimization objective function, where is the label value, and σ(·) is the Sigmoid function. The training learning rate is set to 1e -3 , the total number of training rounds is 300, and the learning rate is decayed by ×0.1 at the 100th and 200th rounds. The gradients g1 and g2 of w1 and w2 are obtained by using the Adam optimizer to backpropagate formula (3), and the weights are updated using the gradients.

[0038] Step 4, test the model using the test set of the open source data set, and get the precision and recall of its classification. Table 1 is the precision and recall of the labels 0 and 1 on the test set.

[0039] Table 1 Precision and recall

[0040] Precision (label 0) Recall (label 0) Precision (label 1) Recall (label 1) 0.97 0.90 0.79 0.95

Claims

1. A weight training method for multi-point hyperspectral binary classification, characterized in that, The device based on the composite dielectric gate structure can sequentially complete linear response and harmonic nonlinear operation of light, and the weight training method comprises the following steps: (1) establishing a preliminary model according to the linear response and harmonic nonlinear operation completed by the device; the preliminary model is: wherein M is the number of hyperspectral data bands, w i is the i-th value of the weight parameter w, y=kx+b is the linear response of the device to a single pixel point x of the hyperspectral image, y i is the i-th band value of the linear response y, y out is the output of the single pixel point of the hyperspectral image through the device; (2) considering the actual limitation of the device: only positive weight can be stored, adding a dual operation to the preliminary model according to the requirements of the classification task, and applying a constraint condition to obtain a classification model; wherein, the adding of the dual operation is specifically: using two groups of weight values w1 and w2 to calculate the linear response y respectively, and subtracting the obtained values: the constraint condition applied to the preliminary model is specifically: applying a positive constraint to the weight, that is: (3) using an open source hyperspectral classification data set, adding noise to the training data and interpolating, binarizing the label, and finally using gradient descent method to update the weight of the classification model until convergence. 2.The weight training method for multi-point hyperspectral binary classification of claim 1, wherein, The device based on the composite dielectric gate structure comprises a composite dielectric gate photosensitive detector and a composite dielectric gate transistor formed above the same P-type semiconductor substrate; the composite dielectric gate photosensitive detector is provided with a first bottom insulating dielectric layer, a first floating gate, a first top insulating dielectric layer and a first control gate above the substrate in sequence, and is provided with a source and a drain on both sides in the substrate; the composite dielectric gate transistor is provided with a second bottom insulating dielectric layer, a second floating gate, a second top insulating dielectric layer and a second control gate above the substrate in sequence, and is provided with a source and a drain on both sides in the substrate; the source of the composite dielectric gate photosensitive detector is shared with the source of the composite dielectric gate transistor.

3. The weight training method for multi-point hyperspectral binary classification according to claim 1, wherein, In the step (3), the noise is added to the training data in sequence, which is specifically Gaussian noise, impulse noise and strip noise.

4. The weight training method for multi-point hyperspectral binary classification according to claim 1, wherein, After the step (3), the test set of the open source data set is used to test the classification model to obtain the precision and recall rate of the classification.

Images