Method, device and equipment for expanding near-ground hyperspectral data of camouflage target

Abstract

The invention provides a method, a device and equipment for expanding near-ground hyperspectral data of a camouflage target, and relates to the technical field of hyperspectral data expansion. According to the method, the spectral distinction degree of the camouflage target and the camouflage background in each wave band is calculated, then the weight value in positive correlation with the spectral distinction degree is determined, and in the training process of the automatic encoder network model, convergence judgment is carried out according to the error of each wave band and the loss function determined by the weight. Therefore, according to the automatic encoder network model provided by the invention, the spectrum distinction degree of each wave band can be used as the basis of the model attention degree to train each wave band, so that the data expansion model pays more attention to the wave bands capable of effectively distinguishing a disguised target from a background in the training process; the generated expansion data can better retain the spectral characteristics of the camouflage target easy to identify on the key wave band, and the similarity and accuracy of the expansion data on the wave band of the camouflage target easy to identify are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral data expansion, and in particular to a method, device and equipment for expanding near-ground hyperspectral data of a camouflaged target. Background Art

[0002] Near-ground hyperspectral imaging technology can perform high-resolution imaging of target areas in continuous spectral bands, usually covering wavelengths from visible light to near-infrared and even longer. This allows the unique spectral characteristics of camouflaged targets to be fully displayed. Due to the high cost of collecting near-ground hyperspectral images of camouflaged targets and the scarcity of samples, which are usually insufficient to train deep learning models to complete subsequent target classification and detection tasks, data augmentation is usually used.

[0003] Data synthesis methods involve synthesizing new, ground-truth labeled data for small sample categories to expand the training dataset. For visible light images, various data synthesis methods exist, such as adding rotation factors, folding, and adding noise. However, these methods are not suitable for hyperspectral object detection because hyperspectral object detection relies on the spectra of pixels, and object spectra are rotationally invariant. Adding rotation factors or folding to the spectra of individual pixels can easily cause feature variation.

[0004] Another method, adding noise, strictly speaking, doesn't increase spectral diversity. It simply expands the sample. However, the fundamental spectrum remains essentially the same after the noise is added, with certain spectral features being crudely repeated. Large amounts of this type of data hinder deep networks from learning effective information, increasing training costs and wasting resources. In summary, existing methods for augmenting near-ground hyperspectral data of camouflaged targets generate samples with low similarity and accuracy compared to the original samples. Summary of the Invention

[0005] The embodiments of the present invention provide a method and device for expanding hyperspectral data of a camouflaged target near the ground, so as to solve the problem of low accuracy of expanded data in the existing hyperspectral data expansion method of a camouflaged target near the ground.

[0006] In a first aspect, an embodiment of the present invention provides a method for expanding near-ground hyperspectral data of a camouflaged target, comprising:

[0007] Obtain spectral datasets of camouflaged targets and camouflaged backgrounds;

[0008] Obtaining the spectral discrimination between the camouflaged target and the camouflaged background in each wavelength band according to the spectral data set of the camouflaged target and the camouflaged background;

[0009] Based on the spectral discrimination of each band, positive correlation processing is performed to determine the weight of the error of each band in the loss function; the error is the difference between the predicted value and the true value of each band;

[0010] Based on the spectral data set and the weights of each band, an autoencoder network model is trained; during the training process, convergence is determined by a loss function determined by the error and weight of each band;

[0011] Based on the trained autoencoder network model, data expansion is performed to obtain the expanded near-ground hyperspectral data of the camouflaged target.

[0012] In one possible implementation, based on the spectral dataset and the weights of each band, the autoencoder network model is trained to include:

[0013] Inputting the spectrum data of each band in the spectrum data set of the disguised target into the automatic encoder network model, and outputting the corresponding reconstructed spectrum data of each band;

[0014] For any band, the error of the band is obtained based on the difference between the input spectral data of the band and the output reconstructed spectral data;

[0015] Based on the weight of the error of the band in the loss function and the error of the band, the loss function of the band is obtained;

[0016] Based on the loss function of each band, the iterative training is repeated until the preset training cutoff condition is met to obtain the trained autoencoder network model.

[0017] In a possible implementation, the loss function includes a weighted mean square error loss function.

[0018] In one possible implementation, the weighted mean square error loss function is obtained based on the following formula:

[0019]

[0020] Among them, MSE weighted represents the weighted mean square error loss function; w m represents the loss function weight of the nth band; x n Represents the spectral data of the nth band input; Represents the reconstructed spectral data output by the nth band.

[0021] In a possible implementation, performing positive correlation processing based on the spectral discrimination of each band to determine the weight of the error of each band in the loss function includes:

[0022] The spectral discrimination of each band is softmax processed to obtain the weight of the error of each band in the loss function, where the sum of the loss function weights of each band is 1.

[0023] In a possible implementation, obtaining the spectral discrimination between the camouflaged target and the camouflaged background in each wavelength band according to the spectral dataset of the camouflaged target and the camouflaged background includes:

[0024] Taking any band as the target band, obtaining the spectral data of the camouflaged target and the camouflaged background in the target band and its adjacent bands;

[0025] Based on the spectral data of the camouflaged target in the target band and its adjacent bands, the neighborhood spectral vector of the camouflaged target in the target band is obtained;

[0026] Based on the spectral data of the camouflage background in the target band and its adjacent bands, the neighborhood spectral vector of the camouflage background in the target band is obtained;

[0027] Based on the neighborhood spectrum vectors of the camouflaged target and the camouflaged background in the target band, the neighborhood spectrum angle cosine values ​​of the camouflaged target and the camouflaged background are obtained;

[0028] The spectral discrimination between the camouflaged target and the camouflaged background in the target band is obtained by subtracting the cosine value of the neighborhood spectral angle from 1.

[0029] In a possible implementation, obtaining the neighborhood spectral angle cosine values ​​of the camouflaged target and the camouflaged background based on the neighborhood spectral vectors of the camouflaged target and the camouflaged background in the target band includes:

[0030] The spectral angle cosine is determined based on the following formula

[0031]

[0032] Among them, cosθ(λ n ) represents the cosine value of the neighborhood spectral angle of the camouflaged target and the camouflaged background in the target band; λ n Indicates the target band; Represents the neighborhood spectrum vector of the camouflage background in the target band; Represents the neighborhood spectrum vector of the camouflaged target in the target band.

[0033] In a possible implementation, obtaining a spectral dataset of a camouflaged target and a camouflaged background includes:

[0034] Acquiring a plurality of near-ground hyperspectral images, wherein the images include camouflaged targets and camouflaged backgrounds;

[0035] The spectral data of the camouflaged target and the spectral data of the camouflaged background in each near-ground hyperspectral image are extracted to obtain the spectral dataset of the camouflaged target and the spectral dataset of the camouflaged background.

[0036] In a second aspect, an embodiment of the present invention provides a device for expanding near-ground hyperspectral data of a camouflaged target, comprising:

[0037] An acquisition module is used to obtain the spectral dataset of the camouflaged target and the camouflaged background;

[0038] A discrimination obtaining module is used to obtain the spectral discrimination between the camouflaged target and the camouflaged background in each band according to the spectral data set of the camouflaged target and the camouflaged background;

[0039] A weight determination module, configured to perform positive correlation processing based on the spectral discrimination of each band to determine the weight of the error of each band in the loss function; the error is the difference between the predicted value and the true value of each band;

[0040] A training module is used to train an autoencoder network model based on the spectral data set and the weights of each band; during the training process, convergence is determined by a loss function determined by the error and weight of each band;

[0041] The expansion module is used to perform data expansion based on the trained autoencoder network model to obtain the expanded near-ground hyperspectral data of the camouflaged target.

[0042] In a third aspect, an embodiment of the present invention provides a device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method in the first aspect or any possible implementation of the first aspect is implemented.

[0043] The present invention calculates the spectral discrimination between the camouflaged target and the camouflaged background in each band, and then determines a weight value that is positively correlated with the spectral discrimination. During the training process of the autoencoder network model, the loss function determined by the error and weight of each band is used to determine convergence. In this way, the autoencoder network model provided by the present invention can use the spectral discrimination of each band as the basis for the model's attention level to train each band. This allows the data augmentation model to pay more attention to bands that can effectively distinguish the camouflaged target from the background during training. The generated augmented data can better retain the spectral characteristics that easily identify the camouflaged target in key bands, and improve the similarity and accuracy of the augmented data in bands that easily identify the camouflaged target. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a diagram illustrating an application scenario of the method for expanding near-ground hyperspectral data of a camouflaged target provided by an embodiment of the present invention;

[0045] Figure 2 This is a flowchart of the implementation of the method for expanding near-ground hyperspectral data of a camouflaged target provided by an embodiment of the present invention;

[0046] Figure 3 Schematic diagram of the automatic encoder network model structure provided by an embodiment of the present invention;

[0047] Figure 4 This is a flowchart of a method for expanding near-ground hyperspectral data for a grassy camouflage target provided by an embodiment of the present invention;

[0048] Figure 5 This is a visible light image containing a grass camouflage target provided by an embodiment of the present invention;

[0049] Figure 6 This is a hyperspectral image containing a grass camouflaged target provided by an embodiment of the present invention;

[0050] Figure 7 Schematic diagram of the spectral reflectance of various camouflaged targets and grass provided by an embodiment of the present invention;

[0051] Figure 8 Schematic diagram of the spectral reflectance of target A before and after expansion provided by an embodiment of the present invention;

[0052] Figure 9 1 is a schematic diagram of the spectral reflectance of target B before and after expansion provided by an embodiment of the present invention;

[0053] Figure 10 1 is a schematic diagram of the spectral reflectance of the C target before and after expansion provided by an embodiment of the present invention;

[0054] Figure 11 Schematic diagram of spectral reflectance of D target before and after expansion provided by an embodiment of the present invention;

[0055] Figure 12 It is a structural diagram of a near-ground hyperspectral data expansion device for camouflaged targets provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0057] Traditional optical detection methods can only obtain information about the target in a limited number of wavelengths, making it difficult to capture the subtle yet critical spectral differences between the camouflaged target and the camouflaged background (such as grass). Near-ground hyperspectral imaging technology, however, can capture high-resolution images of the target area across a continuous spectral band, typically covering wavelengths from visible light to near-infrared and even longer. This allows the unique spectral characteristics of camouflaged targets hidden in the grass to be fully revealed. This is because different camouflage materials and structures, as well as their interaction with the surrounding grass and vegetation, will exhibit complex and varying reflection and absorption characteristics across various spectral bands.

[0058] However, the practical application of hyperspectral data faces significant challenges. First, acquiring high-quality near-ground hyperspectral data is costly, requiring not only specialized and expensive hyperspectral imaging equipment but also significant human and material resources to collect data under diverse environmental conditions. For example, to comprehensively capture the various scenarios in which grassland camouflage targets may appear, multiple data acquisitions are required under varying lighting angles (e.g., early morning oblique light, midday direct sunlight, evening backlight), climate conditions (sunny, overcast, light rain), and grassland growth stages (lush in spring and summer, withered in autumn). Second, even after this arduous data collection process, the amount of raw data obtained remains relatively limited, making it difficult to meet the massive and diverse data requirements of modern data analysis techniques such as deep learning. Furthermore, grassland camouflage targets are made of a variety of materials, ranging from artificially applied camouflage materials to specially crafted biomimetic camouflage fabrics, and their spectral response functions are complex and highly variable. This urgently requires an efficient data augmentation method that can not only simulate spectral variations under various real-world conditions, but also generate rich and diverse data that reflects the characteristics of real scenes based on limited raw data, so as to improve the accuracy and reliability of detecting and identifying grassland camouflaged targets based on near-ground hyperspectral data. In summary, the development of near-ground hyperspectral data augmentation methods for camouflaged targets is of extremely important practical significance and has become a key link in promoting technological progress in related fields.

[0059] Figure 1 This is an application scenario diagram of the method for expanding near-ground hyperspectral data of camouflaged targets provided by an embodiment of the present invention. Figure 1 As shown in the figure, a ground object (i.e., a camouflaged target) is placed against a camouflaged background (e.g., grass). An imaging spectrometer images the camouflaged target and collects hyperspectral data. By analyzing the differences in hyperspectral characteristics between the target and the background, the camouflaged target can be identified. Accurately identifying the type of camouflaged target requires deep learning using a large number of data samples.

[0060] Through the above analysis, the current near-ground hyperspectral data expansion for grass camouflage targets has the following problems:

[0061] ① Traditional target near-ground hyperspectral data expansion is mainly through light perturbation or simply adding noise. Although the operation method is simple, no actual meaningful samples are obtained.

[0062] ②The current spectral expansion method based on probabilistic statistical models is difficult to learn deep spectral features and cannot effectively describe complex spectral diversity.

[0063] While the generative model based on the autoencoder can effectively expand the sample, it does not utilize the prior spectral characteristics of specific targets. This invention, however, uses the near-ground spectral characteristics of grass-camouflaged targets as prior knowledge and improves the AE loss function to make the expansion method more scientific and effective.

[0064] This embodiment of the present invention uses a data generation method to address small-sample learning. A sample generator is constructed using a generative autoencoder network model to address the problem of insufficient target samples. Furthermore, based on an analysis of the spectral characteristics of grass-camouflaged targets, an improved loss function is used to ensure the similarity between the generated samples and the original samples.

[0065] Figure 2 This is a flowchart of the implementation of the method for expanding near-ground hyperspectral data of a camouflaged target provided by an embodiment of the present invention; Figure 2 The embodiment of the present invention provides a method for expanding near-ground hyperspectral data of a camouflaged target, comprising:

[0066] Step 201, obtaining a spectral dataset of a camouflaged target and a camouflaged background;

[0067] For example, the camouflaged target may be camouflage netting, camouflage paint, etc. For example, the camouflaged background may be grass. Typically, the camouflaged target and the camouflaged background are highly similar, making them difficult to distinguish with the naked eye, thereby achieving the purpose of camouflage. The specific types of camouflaged targets and camouflaged backgrounds are not limited herein.

[0068] Exemplarily, the spectral dataset includes multiple sets of spectral data. Each set of spectral data includes spectral indicators within a predetermined wavelength band, such as reflectivity, transmittance, or emissivity within the predetermined wavelength band. The following example illustrates how to obtain a spectral dataset.

[0069] In a possible implementation, obtaining a spectral dataset of a camouflaged target and a camouflaged background includes:

[0070] Step 211: Acquire multiple near-ground hyperspectral images, wherein the images include camouflaged targets and camouflaged backgrounds;

[0071] In some embodiments, an imaging spectrometer is used to acquire near-ground hyperspectral image data containing camouflaged targets. For example, data collection can be performed using a land-based platform (portable, field-based tripod-mounted) equipped with a hyperspectral imaging device. A near-ground hyperspectral imaging spectrometer with an appropriate wavelength range is selected, ensuring high spectral resolution, wide spectral coverage (preferably from visible light to near-infrared), and good near-ground imaging performance to clearly capture subtle features of grass and camouflaged targets.

[0072] In some embodiments, data collection is performed under a variety of environmental conditions to obtain multiple near-ground hyperspectral images. For example, data collection is performed under a variety of environmental conditions, including different weather conditions (sunny, cloudy, overcast, light rain, etc.), different time nodes (early morning, morning, noon, afternoon, evening) and different grass growth stages (germination period, lush period, withered period, etc.). During collection, the height, angle, field of view and other parameters of the imaging spectrometer are strictly controlled to ensure the consistency and comparability of the data. For example, the imaging spectrometer is fixed on a height-adjustable bracket, the height is adjusted from 0.5 meters to 1.5 meters at intervals of 0.2 meters, the angle is adjusted from 0° to 360°, and the shooting is rotated at intervals of 30° to ensure full coverage of the observation situations that may occur in grass camouflage targets.

[0073] In some embodiments, the collected raw hyperspectral image data is preprocessed so that the data can truly reflect the reflectivity characteristics of the target and background. It should be noted that the radiation brightness collected by the detector is an intensity unit and cannot reflect the reflectivity characteristics of the object itself. If you want to compare the spectra of objects under different conditions, you need to perform radiometric calibration on the captured hyperspectral data and convert it into the reflectivity of the object. For example, the whiteboard method can be used, and the reflectivity calculation formula is the pixel brightness (Digital Number, DN) value of the object divided by the DN value of the whiteboard, and then multiplied by the reflectivity of the whiteboard. The reflectivity of the whiteboard is a constant.

[0074] In step 2012, the spectral data of the camouflaged target and the spectral data of the camouflaged background are extracted from each near-ground hyperspectral image to obtain a spectral dataset of the camouflaged target and a spectral dataset of the camouflaged background.

[0075] In some embodiments, spectrum analysis software (such as Matlab) is used to extract spectrum curves of the grass camouflage target and the pure grass background.

[0076] For example, by comparing and analyzing the absorption peaks and reflectance valleys of the two in different bands, we can identify the key spectral bands that can effectively distinguish the target from the background. For example, it can be found that the reflectivity of the camouflaged target in a certain near-infrared band is significantly higher than that of the surrounding grass, while the absorption characteristics in a certain visible light band are very different. These characteristics will provide important basis for subsequent data expansion and model training.

[0077] It should be noted that for camouflaged targets, we aim to achieve better fitting results in bands where the camouflaged target is most distinguishable from the camouflaged background. This means that we place greater emphasis on bands where the camouflaged target is most distinguishable from the camouflaged background. This is achieved by designing fitting weights for different bands in the loss function. The following steps first illustrate how to obtain spectral discrimination.

[0078] Step 202: obtaining the spectral discrimination between the camouflaged target and the camouflaged background in each wavelength band based on the spectral dataset of the camouflaged target and the camouflaged background;

[0079] For example, the spectrum is composed of light of different wavelengths. To facilitate analysis and research, the entire spectrum is usually divided into several specific wavelength intervals, which are called bands.

[0080] For example, spectral discrimination is a metric that measures the degree of difference in spectral characteristics between a camouflaged target and its background within a certain wavelength band. Generally, this difference is quantified by calculating specific statistics or employing specific algorithms. For example, the difference between the mean values ​​or the ratio of the standard deviations of the spectral reflectance of the camouflaged target and its background within a certain wavelength band can be calculated. Alternatively, more complex algorithms such as spectral angle mapping and spectral information divergence can be used to obtain a numerical value that reflects the magnitude of the spectral difference between the two. For example, a larger value indicates a higher spectral discrimination between the camouflaged target and its background within that wavelength band, meaning it is easier to distinguish the camouflaged target from the background. Conversely, a smaller value indicates a lower discrimination, making it more difficult to distinguish the camouflaged target from the background.

[0081] In step 202, the spectral data sets of the camouflaged target and the camouflaged background are analyzed and processed to determine the degree of distinguishability between the two within different spectral bands, providing a basis for the next step of weighting the loss function.

[0082] For example, in the previous step, the n-dimensional camouflage background spectrum T[T 1 ,T 2 ,…,T n ]. The n dimension here represents n bands.

[0083] For example, multiple sets of camouflaged targets are obtained in the previous step. For example, the k sets of camouflaged target spectra D1[D1 1 ,D1 2 ,…,D1 n ],D2[D2 1 ,D2 2 ,…,D2 n ],…,D k [D k 1 ,Dk 2 ,…,D k n ].

[0084] In a possible implementation, obtaining the spectral distinguishability between the camouflaged target and the camouflaged background in each wavelength band according to the spectral dataset of the camouflaged target and the spectral dataset of the camouflaged background includes:

[0085] Step 2021: Taking any wavelength band as a target wavelength band, obtaining spectral data of the camouflaged target and the camouflaged background in the target wavelength band and its adjacent wavelength bands;

[0086] For example, m Indicates the target band of the mth band. Exemplarily, the adjacent bands of the target band may include λ m-1 and λ m+1 .

[0087] Step 2022: Obtaining a neighborhood spectrum vector of the camouflaged target in the target band based on the spectrum data of the camouflaged target in the target band and its adjacent bands;

[0088] For example, specific to a particular band λ m , the neighborhood spectrum vector of the camouflaged target in this band is [D λm-1 ,D λm ,D λm+1 ]. For example, since the first and last two bands (the 1st and nth bands) do not have complete neighborhood information, the neighborhood on one side can be used to calculate the discrimination. For example, the neighborhood spectrum vector of the first band is [D λ1 ,D λ2 ]; the neighborhood spectrum vector of the nth band is [D λn-1 ,D λn ].

[0089] Step 2023: Obtaining a neighborhood spectrum vector of the camouflage background in the target band based on the spectrum data of the camouflage background in the target band and its adjacent bands;

[0090] For example, specific to a particular band λ m , the neighborhood spectrum vector of the camouflage background in this band is [T λm-1 ,T λm ,T λm+1 ]. For example, since the first and last two bands (the 1st and nth bands) do not have complete neighborhood information, the neighborhood on one side can be used to calculate the discrimination. For example, the neighborhood spectrum vector of the first band is [T λ1 ,T λ2 ]; the neighborhood spectrum vector of the nth band is [T λn-1 ,T λn ].

[0091] Step 2024: Obtaining the neighborhood spectral angle cosine values ​​of the camouflaged target and the camouflaged background based on the neighborhood spectral vectors of the camouflaged target and the camouflaged background in the target band;

[0092] It should be noted that the spectral angle cosine value measures the similarity between two spectral vectors by calculating the cosine of the angle between them. This is based on the concept that in high-dimensional spectral space, spectra can be viewed as vectors, and the angle between these vectors reflects the differences in their spectral characteristics. A smaller angle indicates greater similarity between the two spectra, while a larger angle indicates greater spectral difference.

[0093] In some embodiments, obtaining the neighborhood spectral angle cosine values ​​of the camouflaged target and the camouflaged background based on the neighborhood spectral vectors of the camouflaged target and the camouflaged background in the target band includes:

[0094] The spectral angle cosine is determined based on the following formula

[0095]

[0096] Among them, cosθ(λ n ) represents the cosine value of the neighborhood spectral angle of the camouflaged target and the camouflaged background in the target band; λ n Indicates the target band; Represents the neighborhood spectrum vector of the camouflage background in the target band; Represents the neighborhood spectrum vector of the camouflaged target in the target band.

[0097] Step 2025: Subtract the cosine value of the neighborhood spectral angle from 1 to obtain the spectral discrimination between the camouflaged target and the camouflaged background in the target band.

[0098] For example, the camouflaged target and the camouflaged background are at wavelength λ n The spectral discrimination of the two is 1-λ n The spectral angle cosine of the neighborhood.

[0099] In some embodiments, if the spectral expansion task requires simultaneously expanding the spectra of k sets of camouflaged targets, steps 221-225 can be repeated for each of the k sets of camouflaged target spectra to obtain k parameter vectors, each representing the spectral discrimination of a different wavelength band. These k parameter vectors can be averaged to obtain an average parameter vector for subsequent loss function design tasks. It should be noted that the hyperspectral similarity of the same camouflaged object under different environmental conditions is generally high. Therefore, the above average parameter vector approach is particularly suitable for expanding multiple sets of spectra of the same camouflaged object.

[0100] Step 203: Based on the spectral discrimination of each band, a positive correlation process is performed to determine the weight of the error of each band in the loss function; the error is the difference between the predicted value and the true value of each band;

[0101] It should be noted that the loss function here is used for subsequent training of the autoencoder network model. The loss function is a function of the error for each band. Here, the error for a particular band represents the difference between the predicted value and the true value for that band. During the subsequent training process, the error weights for each band are different. The specific weights are determined in step 203.

[0102] Exemplarily, the weight of the error of any band in the loss function is positively correlated with the spectral discrimination of the band.

[0103] In one possible implementation, positive correlation processing is performed based on the spectral discrimination of each band, and the weight of the error of each band in the loss function is determined, including: performing softmax processing on the spectral discrimination of each band to obtain the weight of the error of each band in the loss function, wherein the sum of the loss function weights of each band is 1.

[0104] For example, the spectral discrimination obtained in each band is subjected to softmax processing, and these spectral discriminations are converted into a probability distribution form, so that a parameter vector [w1,w2,…,w n ], and the sum of its parameters is 1.

[0105] The following describes the training and data augmentation of the data augmentation model.

[0106] In some embodiments, before step 204 , the spectral dataset of the disguised target may be divided into a training set and an expansion set according to a preset ratio.

[0107] For example, the spectral dataset of the disguised target is randomly divided into two parts: training samples and samples to be expanded. The training set is used to train the data expansion model, and the expansion set is used for data expansion after model training.

[0108] Exemplarily, the spectral data set of the camouflaged target here may be spectral data of the same camouflaged target under different conditions.

[0109] For example, a reasonable partitioning ratio is usually set based on the total amount of data and actual needs. For example, 70%-80% of the data can be divided into training samples for building the autoencoder network model; the remaining 20%-30% is used as the to-be-expanded samples for subsequent generation of augmented data. Furthermore, random seeding technology can be used to ensure the randomness and repeatability of the partitioning. That is, if the same random seed number is given each time the data is partitioned, the same combination of training samples and to-be-expanded samples can be obtained, facilitating subsequent comparative experiments and model optimization.

[0110] The embodiments of the present invention utilize the same data set for both training and expansion sets, ensuring data consistency and comparability throughout the entire data processing and model training process. Both the training and expansion sets originate from the same original dataset, divided according to a preset ratio. This ensures that data from different stages of model training and evaluation share a common foundation and context, avoiding bias and errors caused by differing data sources and making model training and evaluation results more reliable and accurate.

[0111] Step 204 : Based on the spectral data set and the weights of each band, an autoencoder network model is trained; during the training process, convergence is determined by a loss function determined by the error and weight of each band.

[0112] Exemplarily, a pre-built autoencoder network model is trained based on the spectral data of each band in the spectral dataset of the disguised target;

[0113] Exemplarily, the weight of any band error in the loss function is positively correlated with the spectral discrimination of the band;

[0114] The autoencoder network model, also known as autoencoder (AE), is a feature learning method under self-supervision mode. Its learning strategy can be abstracted as an optimization problem to minimize the reconstruction error. Figure 3 : is a schematic diagram of the structure of the automatic encoder network model provided by an embodiment of the present invention; Figure 3 As shown, a simple model of AE has two main components, namely encoder and decoder.

[0115] The encoder maps the input to a hidden layer subspace through convolution operations, common in deep networks, to form hidden layer features. After obtaining the hidden layer representation through the encoding operation, the decoder uses the hidden layer representation to inversely reconstruct the input. Typically, when using hidden layer values, the final form of AE learning is a reduced-dimensional representation. When using output layer values, the AE learning result is the reconstructed data information. The sample generator uses the output layer values ​​to generate new samples with similar target spectral characteristics. Throughout the training process, the output pixels are identically mapped to the input pixels.

[0116] In some embodiments, the pre-built autoencoder network model includes an input layer, multiple hidden layers, and an output layer. For example, the number of nodes in the input layer corresponds to the number of bands in the hyperspectral data, and the hidden layers use an appropriate number of neurons and activation functions (e.g., ReLU functions) to learn the intrinsic feature representations of the data. The output layer has the same number of nodes as the input layer and is designed to reconstruct the input data.

[0117] For example, assuming that the spectrum vector is n-dimensional data, the network structure is designed as follows:

[0118] 1. Encoder part:

[0119] 1.1. Input layer: accepts n-dimensional spectral data, that is, it has n neurons.

[0120] 1.2. Hidden layer 1: Fully connected layer, the number of neurons is n / 2, using ReLU as the activation function to introduce nonlinear transformation, the formula is

[0121] h1=ReLU(W1x+b1)

[0122] where x is the input spectrum, W1 is the weight matrix, and b1 is the bias vector.

[0123] 1.3. Hidden layer 2: fully connected layer, the number of neurons is n / 4, and the ReLU activation function is also used. The formula is

[0124] h2=ReLU(W2h1+b2)

[0125] 1.4. Latent Representation Layer: Fully connected layer with n / 8 neurons. The output is the encoded low-dimensional representation z = W3h2 + b3.

[0126] 2. Decoder

[0127] 2.1. Hidden layer 3: Fully connected layer with n / 4 neurons. The input is the potential representation. The RLU activation function is used, h3 = ReLU(w4z+b4).

[0128] 2.2. Hidden layer 4: fully connected layer, the number of neurons is n / 2, h4=ReLU(W5h3+b5).

[0129] 2.3. Output layer: Fully connected layer, the number of neurons is, using linear activation function (because we want to output the original numerical range of the spectrum), outputting the reconstructed spectrum

[0130] The AE model designed in the embodiment of the present invention is a simple AE model, and other variations and optimization methods can be used to improve the model. The required model structure includes but is not limited to the above model.

[0131] It should be noted that the pre-built autoencoder network model is trained based on the spectral data of each band in the spectral dataset of the disguised target. The trained autoencoder network model can output the reconstructed spectral data of each band.

[0132] The above describes the pre-built autoencoder network model. The following describes the loss function of the autoencoder network model.

[0133] A loss function is used in machine learning and deep learning to measure the difference between a model's predictions and the true labels. The loss function compares the model's predictions with the actual true values ​​and calculates a numerical value to represent the degree of discrepancy. A smaller value indicates that the model's predictions are closer to the true values, and the model's performance is better; conversely, a larger value indicates that the model's predictions are worse. Essentially, it quantifies the degree of error in the model's predictions, providing a clear goal and direction for model training and optimization.

[0134] In some embodiments, the autoencoder network model inputs spectral data of a certain band (which can be used as the true value) and outputs spectral data reconstructed from that band (which can be used as the predicted value). Exemplarily, the loss function is a function of the true value, the predicted value, and the wavelength. Furthermore, the loss function is also configured with a weight related to the wavelength.

[0135] In some embodiments, the loss function weight of any band is positively correlated with the spectral discrimination of that band. For example, the higher the spectral discrimination of a band, the greater its weight in the loss function, and vice versa. For another example, in order to enable the model to better learn and utilize information from different bands, a weight is assigned to the loss calculation for each band. For example, in a material classification model based on spectral data, different weights may be assigned to the prediction errors of different bands to emphasize the importance of certain bands to the classification results. The following describes the model training process in conjunction with the loss function.

[0136] In one possible implementation, based on the spectral dataset and the weights of each band, the autoencoder network model is trained to include:

[0137] Step 2041: inputting the spectrum data of each band in the spectrum data set of the disguised target into the autoencoder network model, and outputting the corresponding reconstructed spectrum data of each band;

[0138] Step 2042 : For any wavelength band, based on the difference between the input spectrum data and the output reconstructed spectrum data of the wavelength band, obtain the error of the wavelength band.

[0139] For example, the input spectral data can be used as the true value, and the output reconstructed spectral data can be used as the predicted value. Furthermore, based on the difference between the true value and the predicted value of a certain band, the error of the band can be determined.

[0140] Step 2043: Obtain the loss function of the band based on the weight of the error of the band in the loss function and the error of the band.

[0141] The following describes the traditional mean square error loss function. For the input original spectral data x=(x1,x2,…,x n ) and spectral data reconstructed by the autoencoder The calculation formula of the mean square error loss function is:

[0142]

[0143] Considering that features in different parts of the spectrum may have different importance, the mean squared error loss function can be weighted. For example, in addition to the commonly used mean squared error (MSE) loss function, a loss function can be designed based on the physical properties of the grass camouflage target spectrum. For example, in the spectrum of a grass camouflage target, the features in the wavelength region after 610nm are more important than those in other regions. The reconstruction error in these important regions can be given a higher weight.

[0144] Exemplarily, the loss function is improved: the weights are changed to the parameter vector obtained above.

[0145] In a possible implementation, the loss function includes a weighted mean square error loss function.

[0146] In one possible implementation, the weighted mean square error loss function is obtained based on the following formula:

[0147]

[0148] Among them, MSE weighted represents the weighted mean square error loss function; w nrepresents the loss function weight of the nth band; x n Represents the spectral data of the nth band input; Represents the reconstructed spectral data output by the nth band.

[0149] Step 2044: Repeat the iterative training based on the loss function of each band until a preset training cutoff condition is met, thereby obtaining a trained autoencoder network model.

[0150] For example, training samples were fed into an autoencoder, and the network parameters were continuously adjusted using a backpropagation algorithm. After 50 rounds of iterative training, and then increasing the number of iterations based on the convergence of the loss function, the network was able to accurately capture the characteristic patterns of hyperspectral data, possess good generalization capabilities, and effectively reconstruct similar data.

[0151] Step 205 : performing data expansion based on the trained autoencoder network model to obtain expanded near-ground hyperspectral data of the camouflaged target.

[0152] In some embodiments, the samples to be expanded are input into a trained model and the expanded hyperspectral data is output.

[0153] In some embodiments, the expanded set described above may be input into a trained autoencoder network model for data expansion.

[0154] In some embodiments, the samples to be augmented are fed one by one into a fully trained autoencoder network model. Based on the learned feature representations, the model encodes and decodes the input samples, outputting augmented samples with similar spectral characteristics to the original samples but with subtle differences. These differences arise from the network's learning and simulation of the data distribution during training, and can, to a certain extent, reflect the diversity found in real-world scenarios.

[0155] In some embodiments, steps 203 - 205 are iterated until the number of samples required for the subsequent task is met.

[0156] For example, after each round of expansion is completed, the total number of samples after the current expansion is counted and compared with the number of samples required for the preset subsequent tasks (such as the minimum number of samples required for deep learning classification model training, the data volume requirements for high-precision target recognition algorithms, etc.). If the requirements have not been met, the data partitioning, model training, and sample expansion steps are repeated to continuously enrich the sample set. During multiple iterations, the model parameters (such as hidden layer structure, learning rate, etc.) and the partition ratio can be adjusted in a timely manner to optimize the expansion effect and improve the quality and diversity of the expanded data, ultimately obtaining sufficient hyperspectral data samples to meet the needs of various complex grassland camouflage target recognition tasks.

[0157] The embodiment of the present invention determines the weight of the loss function of the corresponding band based on the spectral discrimination between the camouflaged target and the camouflaged background in each band; based on the weighted improved loss function, an autoencoder network model is trained to expand the near-ground hyperspectral data of the camouflaged target. The embodiment of the present invention determines the spectral discrimination between the camouflaged target and the camouflaged background in each band, and based on the discrimination of a certain band, sets a positive correlation weight for the loss function of the data expansion model in that band, and sets different loss function weights in different bands, so that the data expansion model pays more attention to those bands that can effectively distinguish the camouflaged target from the background during the training process. This loss function weighting method enables the generated expanded data to better retain the unique spectral characteristics of the camouflaged target in the key bands, thereby improving the similarity and accuracy of the expanded data.

[0158] The data expansion method of the embodiment of the present invention has lower cost than the actual collection method, has improved accuracy compared to the noise addition method and the transformation method, and is more in line with the actual situation.

[0159] The following is a comprehensive embodiment to illustrate the technical concept of the present invention. Figure 4 This is a flowchart of a near-ground hyperspectral data expansion method for a grass camouflage target provided by an embodiment of the present invention; Figure 4 , methods include:

[0160] ①Use an imaging spectrometer to obtain near-ground hyperspectral image data containing grass camouflage targets.

[0161] ② Spectral characteristics analysis of near-ground grass camouflage targets: Analyze the spectral differences between the target to be expanded and the background (not limited to grass). In the subsequent design of the model loss function, assign greater weight to the bands with low spectral similarity and high discrimination between the camouflage target and grass. The specific steps are described in detail below.

[0162] ③ Randomly divide the target hyperspectral data into two parts: training samples and samples to be expanded.

[0163] ④ Use training samples to train the autoencoder network model.

[0164] ⑤ Input the samples to be expanded into the trained model and output the expanded hyperspectral data.

[0165] ⑥ Repeat steps ③-⑤ until the number of samples required for subsequent tasks is met.

[0166] Embodiments of the present invention provide a stable and effective method for expanding near-ground hyperspectral data for grass-camouflaged targets. This method first analyzes the spectral characteristics of the near-ground grass-camouflaged targets and then randomly divides the existing samples into training samples and samples to be expanded. The training samples are used to train the designed autoencoder network model, and the loss function of the autoencoder network model is improved based on the spectral characteristics of the near-ground grass-camouflaged targets. The samples to be expanded are then used as input to the trained model, and the resulting output spectrum is the expanded spectrum. By iteratively dividing samples and generating spectral samples, a sufficient number of target spectral samples are obtained to achieve the purpose of expanding the spectral data. This method effectively addresses the problem of the high cost of acquiring current target hyperspectral data, which has limited the application of subsequent classification and detection methods based on deep learning.

[0167] An embodiment of the present invention provides a method for expanding near-ground target spectral samples based on an autoencoder. Compared to traditional spectral data expansion methods, this method can learn deep-level connections between spectral bands, making it a more scientific and effective expansion method. The autoencoder network includes but is not limited to traditional autoencoders, stacked denoising autoencoders, convolutional autoencoders, and variational autoencoders. The AE model designed in this invention is a simple AE model, and other variants and optimization methods can be used to improve the model.

[0168] This embodiment combines target spectral characteristic analysis with improved loss function design for an autoencoder generative model. Taking a near-ground grass camouflage target as an example, the loss function is designed with weighted parameters based on the discriminability between the target and background in different wavelength bands, thereby obtaining a more discriminative extended spectrum for subsequent tasks.

[0169] This paper presents a method for expanding near-ground hyperspectral data for grassland camouflage targets. This method provides data support for deep learning-based classification and detection of grassland camouflage targets, significantly improving subsequent classification and detection results.

[0170] In some embodiments, a reference image is obtained by using an imaging spectrometer. The image has a spatial dimension of 1002×1002 pixels, a spectral dimension of 89, a band interval of 4 nm, and is imaged in the range of 449 nm to 801 nm. The visible light image of the grass camouflage target and the hyperspectral image are as follows: Figure 5 、 Figure 6 shown. Figure 5 This is a visible light image containing a grass camouflage target provided by an embodiment of the present invention; Figure 6 This is a hyperspectral image containing a grass camouflage target provided by an embodiment of the present invention. In the figure, A, B, C, and D represent different camouflage targets.

[0171] According to step ②, the obtained hyperspectral image is subjected to spectral characteristic analysis to obtain the spectral reflectances of A, B, C, and D. Figure 7 Schematic diagram of the spectral reflectance of various camouflaged targets and grass provided by an embodiment of the present invention.

[0172] Calculate the spectral discrimination between the grass and four camouflaged targets (A, B, C, and D). Different targets have different discriminations, and the weights in the loss function are also different. Take the average of the four resulting parameter vectors and use them as the weight parameter in the model's loss function.

[0173] The network structure is set as follows: Input layer: accepts 89-dimensional spectral data, that is, it has 89 neurons. Hidden layer 1: fully connected layer, the number of neurons is 45, using ReLU as the activation function. Hidden layer 2: fully connected layer, the number of neurons is 20, also using ReLU activation function. Latent representation layer (Latent Representation): fully connected layer, the number of neurons is 10, and the output is the encoded low-dimensional representation. Hidden layer 3: fully connected layer, the number of neurons is 20, the input is the latent representation, and the ReLU activation function is used. Hidden layer 4: fully connected layer, the number of neurons is 45. Output layer: fully connected layer, the number of neurons is 89, using a linear activation function (because we want to output the original numerical range of the spectrum), and outputting the reconstructed spectrum.

[0174] According to the subsequent steps, the hyperspectral data of the four camouflaged targets, A, B, C, and D, were randomly divided into training samples and augmented samples in a ratio of 7:3. The learning rate was set to 0.001, the number of epochs was set to 50, and a dropout layer was added to the hidden layer with a dropout probability of 0.4. The training samples were used to train the autoencoder network model (the most basic model) and the loss function was improved. Figures 8 to 11 The spectrum expansion effects of four types of grass camouflage materials, A, B, C, and D, are demonstrated. Figure 8 Schematic diagram of the spectral reflectance of target A before and after expansion provided by an embodiment of the present invention; Figure 9 1 is a schematic diagram of the spectral reflectance of target B before and after expansion provided by an embodiment of the present invention; Figure 10 1 is a schematic diagram of the spectral reflectance of the C target before and after expansion provided by an embodiment of the present invention; Figure 11 3 is a schematic diagram of the spectral reflectance of the D target before and after expansion provided by an embodiment of the present invention.

[0175] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0176] The following are device embodiments of the present invention. For details not fully described therein, reference may be made to the corresponding method embodiments described above.

[0177] Figure 12 The following is a schematic diagram showing the structure of a device for expanding near-ground hyperspectral data of a camouflaged target provided by an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown, which are described in detail as follows:

[0178] like Figure 12 As shown, a device 2 for expanding near-ground hyperspectral data of a camouflaged target includes:

[0179] An acquisition module 21 is used to acquire a spectral dataset of a camouflaged target and a camouflaged background;

[0180] A discrimination obtaining module 22 is configured to obtain the spectral discrimination between the camouflaged target and the camouflaged background in each wavelength band based on the spectral dataset of the camouflaged target and the camouflaged background;

[0181] A weight determination module 23 is configured to perform positive correlation processing based on the spectral discrimination of each band to determine the weight of the error of each band in the loss function; the error is the difference between the predicted value and the true value of each band;

[0182] A training module 24 is configured to train an autoencoder network model based on the spectral dataset and the weights of each band; during the training process, convergence is determined by a loss function determined by the error and weight of each band;

[0183] The expansion module 25 is used to perform data expansion based on the trained autoencoder network model to obtain the expanded near-ground hyperspectral data of the camouflaged target.

[0184] The embodiments of the present invention calculate the spectral discrimination between the camouflaged target and the camouflaged background in each band, thereby determining a weight value that is positively correlated with the spectral discrimination. During the training process of the autoencoder network model, convergence is determined using a loss function determined by the error and weights for each band. In this way, the autoencoder network model provided by the present invention can use the spectral discrimination of each band as the basis for the model's attention to train each band. This allows the data augmentation model to pay more attention to bands that can effectively distinguish the camouflaged target from the background during training. The generated augmented data can better retain the spectral features that easily identify the camouflaged target in key bands, thereby improving the similarity and accuracy of the augmented data in bands that easily identify the camouflaged target.

[0185] An embodiment of the present invention further provides a device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method in the above method embodiment when executing the computer program.

[0186] In the above embodiments, the descriptions of each embodiment have their own focus. For parts not described or recorded in detail in one embodiment, please refer to the relevant descriptions of other embodiments. Unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other. The technical features of different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0187] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for expanding near-ground hyperspectral data of a camouflaged target, characterized in that: include: Obtain spectral datasets of camouflaged targets and camouflaged backgrounds; Obtaining the spectral discrimination between the camouflaged target and the camouflaged background in each wavelength band according to the spectral data set of the camouflaged target and the camouflaged background; Based on the spectral discrimination of each band, positive correlation processing is performed to determine the weight of the error of each band in the loss function; the error is the difference between the predicted value and the true value of each band; Based on the spectral data set and the weights of each band, an autoencoder network model is trained; during the training process, convergence is determined by a loss function determined by the error and weight of each band; Based on the trained autoencoder network model, data expansion is performed to obtain the expanded near-ground hyperspectral data of the camouflaged target.

2. The method for expanding near-ground hyperspectral data of a camouflaged target according to claim 1, characterized in that: Based on the spectral dataset and the weights of each band, the trained autoencoder network model includes: Inputting the spectrum data of each band in the spectrum data set of the disguised target into the automatic encoder network model, and outputting the corresponding reconstructed spectrum data of each band; For any band, the error of the band is obtained based on the difference between the input spectral data of the band and the output reconstructed spectral data; Based on the weight of the error of the band in the loss function and the error of the band, the loss function of the band is obtained; Based on the loss function of each band, the iterative training is repeated until the preset training cutoff condition is met to obtain the trained autoencoder network model.

3. The method for expanding near-ground hyperspectral data of a camouflaged target according to claim 2, characterized in that: The loss function includes a weighted mean square error loss function.

4. The method for expanding near-ground hyperspectral data of a camouflaged target according to claim 3, characterized in that: The weighted mean square error loss function is obtained based on the following formula: Among them, MSE weighted represents the weighted mean square error loss function; w n represents the loss function weight of the nth band; x n Represents the spectral data of the nth band input; Represents the reconstructed spectral data output by the nth band.

5. The method for expanding near-ground hyperspectral data of a camouflaged target according to claim 2, characterized in that: Based on the spectral discrimination of each band, positive correlation processing is performed to determine the weight of the error of each band in the loss function, including: The spectral discrimination of each band is softmax processed to obtain the weight of the error of each band in the loss function, where the sum of the loss function weights of each band is 1.

6. The method for expanding near-ground hyperspectral data of a camouflaged target according to claim 1, characterized in that: According to the spectral data set of the camouflaged target and the camouflaged background, the spectral discrimination between the camouflaged target and the camouflaged background in each band is obtained, which includes: Taking any band as the target band, obtaining the spectral data of the camouflaged target and the camouflaged background in the target band and its adjacent bands; Based on the spectral data of the camouflaged target in the target band and its adjacent bands, the neighborhood spectral vector of the camouflaged target in the target band is obtained; Based on the spectral data of the camouflage background in the target band and its adjacent bands, the neighborhood spectral vector of the camouflage background in the target band is obtained; Based on the neighborhood spectrum vectors of the camouflaged target and the camouflaged background in the target band, the neighborhood spectrum angle cosine values ​​of the camouflaged target and the camouflaged background are obtained; The spectral discrimination between the camouflaged target and the camouflaged background in the target band is obtained by subtracting the cosine value of the neighborhood spectral angle from 1.

7. The method for expanding near-ground hyperspectral data of a camouflaged target according to claim 6, characterized in that: Based on the neighborhood spectrum vectors of the camouflaged target and the camouflaged background in the target band, the neighborhood spectrum angle cosine values ​​of the camouflaged target and the camouflaged background are obtained: The spectral angle cosine is determined based on the following formula Among them, cosθ(λ n ) represents the cosine value of the neighborhood spectral angle of the camouflaged target and the camouflaged background in the target band; λ n Indicates the target band; Represents the neighborhood spectrum vector of the camouflage background in the target band; Represents the neighborhood spectrum vector of the camouflaged target in the target band.

8. The method for expanding near-ground hyperspectral data of a camouflaged target according to claim 1, characterized in that: The obtaining of the spectral dataset of the camouflaged target and the camouflaged background comprises: Acquiring a plurality of near-ground hyperspectral images, wherein the images include camouflaged targets and camouflaged backgrounds; The spectral data of the camouflaged target and the spectral data of the camouflaged background in each near-ground hyperspectral image are extracted to obtain the spectral dataset of the camouflaged target and the spectral dataset of the camouflaged background.

9. A device for expanding near-ground hyperspectral data of a camouflaged target, characterized in that: include: An acquisition module is used to obtain the spectral dataset of the camouflaged target and the camouflaged background; A discrimination obtaining module is used to obtain the spectral discrimination between the camouflaged target and the camouflaged background in each band according to the spectral data set of the camouflaged target and the camouflaged background; A weight determination module, configured to perform positive correlation processing based on the spectral discrimination of each band to determine the weight of the error of each band in the loss function; the error is the difference between the predicted value and the true value of each band; A training module is used to train an autoencoder network model based on the spectral data set and the weights of each band; during the training process, convergence is determined by a loss function determined by the error and weight of each band; The expansion module is used to perform data expansion based on the trained autoencoder network model to obtain the expanded near-ground hyperspectral data of the camouflaged target.

10. A device, characterized in that The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method for expanding near-ground hyperspectral data of a camouflaged target according to any one of claims 1 to 8 is implemented.

Images