Hyperspectral data enhancement method, system and equipment based on mixing conditions and medium

Abstract

The invention provides a hyperspectral data enhancement method, system and equipment based on mixing conditions and a medium, and the method comprises the steps: obtaining an original hyperspectral data set which comprises a mildew grade label; constructing a generative adversarial network model, wherein the generative adversarial network model comprises a generator and a discriminator; a Wasserstein GAN framework with gradient penalty is applied to train the generative adversarial network model, a loss function of the discriminator comprises antagonism loss, gradient penalty loss and auxiliary classification loss, and a loss function of the generator comprises antagonism loss and auxiliary classification loss; generating a synthetic hyperspectral sample according to the specified mildew grade label and the random noise by using the trained generator; combining the synthesized hyperspectral sample with an original sample according to a preset mixing strategy to form an enhanced data set; using the enhanced data set to train a mildew grade classification model; and classifying the tobacco leaves according to the mildew grade classification model.

Description

Technical Field

[0001] This application relates to the fields of data processing and computer vision technology, and in particular to a hyperspectral data enhancement method, system, electronic device and storage medium based on mixed conditions. Background Technology

[0002] With the increasing demand for non-destructive, rapid, and accurate detection in industrial inspection and agricultural product quality control, hyperspectral imaging technology, with its ability to provide rich "spectral fingerprint" information, has been widely used for material composition identification and defect detection. Combining deep learning methods can achieve automated grade discrimination and defect localization; however, these methods heavily rely on large-scale, well-annotated, and class-balanced training data. In many practical scenarios (such as determining the mold grade of tobacco leaves), samples of intermediate grades or specific defects are naturally scarce, resulting in a severe imbalance in training data, which in turn limits the accuracy and robustness of the detection model.

[0003] In related technologies, traditional data augmentation and oversampling methods (such as SMOTE) generate new samples through linear interpolation, but they are difficult to characterize the complex manifolds of hyperspectral data that are high-dimensional, nonlinear, and have fine-grained spectral peaks. Existing GANs still have several bottlenecks in hyperspectral augmentation applications—including the insufficient ability of conventional models to extract multi-scale spectral features (which have both wide and gentle baseline drift and narrow-band absorption peaks), the easy dilution of simple label splicing conditionalization in deep networks leading to weak class control, and the fact that traditional GAN ​​training objectives focus on "realism" rather than direct optimization of downstream classification tasks, resulting in generated samples that appear realistic but contribute little to improving the effectiveness of the classifier. Summary of the Invention

[0004] In view of this, it is necessary to provide a hyperspectral data enhancement method, system, electronic device and storage medium based on mixed conditions, which can at least overcome one of the above defects.

[0005] In a first aspect, embodiments of this application provide a hyperspectral data enhancement method based on mixed conditions, applied to the detection of mold growth in tobacco leaves, the method comprising: Obtain the raw hyperspectral dataset, which includes mold level labels; Construct a generative adversarial network (GAN) model, which includes a generator and a discriminator; The generator employs a hybrid conditional embedding mechanism and a multi-scale channel attention module. The hybrid conditional embedding mechanism includes: concatenating a random noise vector with a mold level embedding vector as global category information guidance during the initial input stage; simultaneously, mapping category labels to intermediate style vectors through a style mapping network and performing hierarchical style modulation at each layer of the generator using an adaptive instance normalization layer; the multi-scale channel attention module includes multiple parallel depthwise separable convolutional branches, each with a different kernel size; the feature maps output by each branch are concatenated along the channel dimension and then weighted by the channel attention module; the discriminator uses an auxiliary classifier architecture, including an adversarial head for judging the authenticity of data and a classification head for predicting mold levels. The generative adversarial network model is trained using the Wasserstein GAN framework with gradient penalty, wherein the loss function of the discriminator includes adversarial loss, gradient penalty loss and auxiliary classification loss, and the loss function of the generator includes adversarial loss and auxiliary classification loss; The trained generator is used to generate synthetic hyperspectral samples based on specified mold level labels and random noise. The synthesized hyperspectral samples are combined with the original samples according to a preset mixing strategy to form an enhanced dataset; The model for classifying mold grades was trained using an augmented dataset. Tobacco leaves are classified according to the mold grade classification model.

[0006] In one embodiment, the hybrid conditional embedding mechanism further includes: concatenating the embedding vector of the random noise vector with the embedding vector of the mold level category label as the initial input of the generator, so as to provide global category information guidance for the generative adversarial network model; The mold level category label is mapped to an intermediate style vector through a style mapping network to achieve hierarchical style modulation. The style mapping network is a multilayer perceptron, which is used to convert the mold level category label into the intermediate style vector. The intermediate style vector is used to modulate the feature map at each layer of the generator.

[0007] In one embodiment, the style mapping network is a multilayer perceptron, which includes at least two fully connected layers; In each processing block of the generator, the adaptive instance normalization layer modulates the feature map using a modulation formula, which is:

[0008] in For the first Feature map of the layer and These represent the channel mean and standard deviation of the feature map, respectively. and To obtain from style vectors Derived scaling factor and bias factor.

[0009] In one embodiment, the loss function of the discriminator includes discriminator adversarial loss, gradient penalty loss, and generator adversarial loss; The expression for the adversarial loss of the discriminator is as follows:

[0010] in To generate samples, For real samples, , These are the scores given by the discriminator's adversarial head to generated and real samples, respectively. The data distribution generated by the generator For the true data distribution, For category labels, For counter-attack losses; The expression for the gradient penalty loss is:

[0011] in, These are the interpolation points between the real sample and the generated sample. For the distribution of interpolation points, The discriminator's adversarial head output is about the interpolation point. gradient, It is the L2 norm. The loss is a gradient penalty. The expression for the generator's adversarial loss is:

[0012] in, To combat loss in generators.

[0013] In one embodiment, the multi-scale channel attention module includes three parallel depthwise separable convolutional branches; Each of the depthwise separable convolutional branches consists of depthwise convolution and pointwise convolution; The multi-scale channel attention module concatenates the feature maps output by each depthwise separable convolutional branch along the channel dimension, and then performs channel attention weighting through the compression and activation module. The compression and activation module includes a global average pooling layer and two fully connected layers, used to calculate the attention weights for each feature channel.

[0014] In one embodiment, the operational expression for the depthwise separable convolution branch in the multi-scale channel attention module is:

[0015] in For the input feature map, The kernel size is Depth convolution, Point-to-point convolution; The formula for concatenating the output feature maps of each branch along the channel dimension is:

[0016] in, These are the output feature maps of the three depthwise separable convolutional branches. This means that these feature maps are concatenated along the channel dimension to form a multi-scale feature representation; The formula for the global average pooling operation is:

[0017] Where u is the input feature map and L is the spectral length; The formula for calculating the channel attention weight is as follows:

[0018] in and For trainable weight matrix, It is the sigmoid activation function. The activation function for the linear rectifier unit; The formula for the weighted processing is:

[0019] in Represents the Hadamard product. The output feature map after attention weighting. This represents the multi-scale feature representation after splicing. The output feature map of the multi-scale channel attention module is passed to the next layer of the network through the adaptive instance normalization layer.

[0020] In one embodiment, the preset mixing strategy includes: The classifier or discriminator adds the synthetic samples to the augmented dataset if the confidence level of the synthetic samples is higher than a preset value.

[0021] Secondly, embodiments of this application provide a hyperspectral data enhancement system based on mixed conditions, the system comprising: The data acquisition module is used to acquire the raw hyperspectral dataset, which includes mold grade labels; A network construction module is used to construct a generative adversarial network (GAN) model, which includes a generator and a discriminator. The generator employs a hybrid conditional embedding mechanism and a multi-scale channel attention module. The hybrid conditional embedding mechanism includes: concatenating a random noise vector with a mold level embedding vector as global category information guidance during the initial input stage; simultaneously, mapping category labels to intermediate style vectors through a style mapping network; and using an adaptive instance normalization layer to perform hierarchical style modulation at each layer of the generator. The multi-scale channel attention module includes multiple parallel depthwise separable convolutional branches, each with a different kernel size. The feature maps output by the depthwise separable convolutional branches are concatenated along the channel dimension and then weighted by the channel attention module. The discriminator uses an auxiliary classifier architecture, including an adversarial head for judging data authenticity and a classification head for predicting mold levels. The network training module is used to train the generative adversarial network model using the Wasserstein GAN framework with gradient penalty. The discriminator's loss function includes adversarial loss, gradient penalty loss, and auxiliary classification loss, while the generator's loss function includes adversarial loss and auxiliary classification loss. The trained generator is used to generate synthetic hyperspectral samples based on specified mold grade labels and random noise. The synthetic hyperspectral samples are then combined with the original samples according to a preset mixing strategy to form an augmented dataset. The augmented dataset is then used to train a mold grade classification model. The classification module is used to classify tobacco leaves according to the mold grade classification model.

[0022] Thirdly, embodiments of this application provide an electronic device, including: processor; Memory used to store processor-executable instructions; The processor is configured to implement the hybrid-condition-based hyperspectral data enhancement method as described in the first aspect when executing the instructions.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that instruct a device to perform the hyperspectral data enhancement method based on mixed conditions as described in the first aspect.

[0024] This application provides a method, system, electronic device, and storage medium for hyperspectral data augmentation based on mixed conditions. By combining global category guidance with hierarchical style modulation through a mixed condition embedding mechanism in the generator, and by effectively capturing the multi-scale spectral features of hyperspectral data with the help of a multi-scale channel attention module, and by using a discriminator with an auxiliary classifier and a WGAN-GP training framework, it can generate synthetic hyperspectral data with high fidelity and distinct category features. This significantly alleviates the problem of insufficient model training caused by sample scarcity and class imbalance, and effectively improves the accuracy, robustness, and generalization ability of the downstream tobacco mold grade classification model. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating a hyperspectral data enhancement method based on mixed conditions, provided as an embodiment of this application.

[0026] Figure 2a The principal component analysis visualization results for a severe mold category provided in an embodiment of this application.

[0027] Figure 2b The principal component analysis visualization results for a mild mold category provided in an embodiment of this application.

[0028] Figure 2c The visualization results of principal component analysis for the normal category provided in an embodiment of this application.

[0029] Figure 2d The visualization result of t-distribution random neighborhood embedding for a severe mold category provided in an embodiment of this application.

[0030] Figure 2e The visualization result of t-distribution random neighborhood embedding for mild mold category provided in an embodiment of this application.

[0031] Figure 2f Visualization results of t-distributed random neighborhood embeddings for normal categories provided in an embodiment of this application.

[0032] Figure 3a The average formation spectrum of a severe mold category provided in one embodiment of this application.

[0033] Figure 3b The average formation spectrum of a mild mold category provided in one embodiment of this application.

[0034] Figure 3c The average generated spectrum of the normal category is provided in one embodiment of this application.

[0035] Figure 4 A schematic diagram of a hyperspectral data enhancement system based on mixed conditions provided in an embodiment of this application.

[0036] Figure 5 A schematic diagram of an electronic device provided in an embodiment of this application.

[0037] Explanation of main component symbols Hyperspectral data enhancement system based on mixed conditions 10 Data Acquisition Module 11 Network Building Module 12 Network Training Module 13 Classification Module 14 Electronic devices 20 Processor 21 Memory 22 Method steps S100-700 Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0039] It should be noted that, in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0040] It should be noted that in the embodiments of this application, the terms "first," "second," etc., are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. Features specified as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0041] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] With the increasing demand for non-destructive, rapid, and accurate detection in industrial inspection and agricultural product quality control, hyperspectral imaging technology, with its ability to provide rich "spectral fingerprint" information, has been widely used for material composition identification and defect detection. Combining deep learning methods can achieve automated grade discrimination and defect localization; however, these methods heavily rely on large-scale, well-annotated, and class-balanced training data. In many practical scenarios (such as determining the mold grade of tobacco leaves), samples of intermediate grades or specific defects are naturally scarce, resulting in a severe imbalance in training data, which in turn limits the accuracy and robustness of the detection model.

[0043] In related technologies, traditional data augmentation and oversampling methods (such as SMOTE) generate new samples through linear interpolation, but they struggle to characterize the complex manifolds of hyperspectral data, which are high-dimensional, nonlinear, and have fine-grained spectral peaks. Although Generative Adversarial Networks (GANs) have shown potential in terms of generation quality, existing GANs still face several bottlenecks in hyperspectral augmentation applications. These include the insufficient ability of conventional models to extract multi-scale spectral features (which have both broad and gentle baseline drift and narrow-band absorption peaks), the tendency of simple label-based conditionalization to be diluted in deep networks, resulting in weak class control, and the fact that traditional GAN ​​training focuses on "realism" rather than direct optimization of downstream classification tasks, leading to generated samples that appear realistic but contribute little to improving classifier effectiveness. Therefore, there is an urgent need for a novel data augmentation method that can maintain the spectral details and overall distribution realism of hyperspectral data while achieving fine-grained, controllable class generation and directly optimizing downstream discriminative performance.

[0044] This application provides a method, system, electronic device, and storage medium for hyperspectral data augmentation based on mixed conditions. By combining global category guidance with hierarchical style modulation through a mixed condition embedding mechanism in the generator, and by effectively capturing the multi-scale spectral features of hyperspectral data with the help of a multi-scale channel attention module, and by using a discriminator with an auxiliary classifier and a WGAN-GP training framework, it can generate synthetic hyperspectral data with high fidelity and distinct category features. This significantly alleviates the problem of insufficient model training caused by sample scarcity and class imbalance, and effectively improves the accuracy, robustness, and generalization ability of the downstream tobacco mold grade classification model.

[0045] Figure 1 This is a flowchart illustrating a hyperspectral data enhancement method based on mixed conditions provided in an embodiment of this application, as shown below. Figure 1The hyperspectral data augmentation method based on hybrid conditions shown includes at least the following steps: S100: Obtain the original hyperspectral dataset; S200: Construct a generative adversarial network (GAN) model; S300: Train the GAN model using the Wasserstein GAN framework with gradient penalty; S400: Apply the trained generator to generate synthetic hyperspectral samples based on specified mold grade labels and random noise; S500: Combine the synthetic hyperspectral samples with the original samples according to a preset mixing strategy to form an augmented dataset; S600: Train a mold grade classification model using the augmented dataset; S700: Classify tobacco leaves according to the mold grade classification model.

[0046] S100: Obtain the raw hyperspectral dataset.

[0047] In this embodiment of the application, the hyperspectral data enhancement method based on mixed conditions includes, in step S100: acquiring the original hyperspectral dataset. The hyperspectral dataset includes mold grade labels. Specifically, obtaining the original hyperspectral dataset includes: acquiring spectral data of tobacco leaf samples using a hyperspectral imager within a predetermined spectral range (e.g., 400 nm–1000 nm or 900 nm–1700 nm, depending on the embodiment), and labeling each sample with a mold level label (e.g., severe mold, mild mold, normal); performing preprocessing on the acquired hyperspectral data, including but not limited to: bad band removal, band selection, spectral domain smoothing filtering, noise suppression (e.g., wavelet denoising or baseline correction), illumination normalization, and optional PCA-based dimensionality reduction or subband selection, to obtain preprocessed hyperspectral samples for training.

[0048] It is understandable that the above-mentioned acquisition equipment, spectral range, preprocessing methods, and dimensionality reduction parameters can be adjusted according to specific applications; the labeling of mold levels can be obtained by manual expert sampling or by combining optical / chemical detection methods; and preprocessing can be implemented on cloud or edge devices to adapt to different deployment needs.

[0049] S200: Construct a generative adversarial network model.

[0050] In this embodiment of the application, the hyperspectral data enhancement method based on mixed conditions includes, in step S200: constructing a generative adversarial network (GAN) model. The GAN model includes a generator and a discriminator.

[0051] The generator employs a hybrid conditional embedding mechanism and a multi-scale channel attention module. The hybrid conditional embedding mechanism includes: concatenating a random noise vector with a mold level embedding vector as global category information guidance during the initial input stage; simultaneously, mapping category labels to intermediate style vectors through a style mapping network and performing hierarchical style modulation at each layer of the generator using an adaptive instance normalization layer; the multi-scale channel attention module includes multiple parallel depthwise separable convolutional branches, each with a different kernel size; the feature maps output by each branch are concatenated along the channel dimension and then weighted by the channel attention module; the discriminator uses an auxiliary classifier architecture, including an adversarial head for judging the authenticity of data and a classification head for predicting mold levels.

[0052] Specifically, constructing a generative adversarial network model includes: designing a generator module, in which the generator concatenates a random noise vector following a standard normal or uniform distribution with a mold grade category vector mapped by an embedding layer as a global conditional input, and maps it to an intermediate style vector w through a multilayer perceptron (Mapping Network) or several fully connected layers; each generation block of the generator first passes through a multi-scale channel attention module (the module consists of at least two or three parallel depthwise separable convolutional branches, with the branch convolutional kernel size selectable in the range of, for example, 3, 5, 7 or kernel length 3–15), concatenates the outputs of each branch in the channel dimension, and then weights them through channel attention (e.g., SE or a compression-excitation structure of two fully connected layers), and then the AdaIN layer performs affine modulation on the channel scale γ(w) and offset β(w) respectively with the style vector w; the discriminator adopts a one-dimensional convolutional network or a one-dimensional residual network structure, and sets an adversarial scoring head (for Wasserstein scoring) and an auxiliary classification head (Softmax output for mold grade prediction).

[0053] It is understood that the specific number of layers, number of channels, convolution kernel size, activation function and normalization method of the generator and discriminator can all be given in the specification as preferred embodiments with multiple numerical ranges to expand the scope of protection; the number of multi-scale branches and attention form can be replaced with functionally equivalent structures according to computational resources and accuracy requirements.

[0054] S300: Train a generative adversarial network model using the Wasserstein GAN framework with gradient penalty.

[0055] In this embodiment, the hyperspectral data augmentation method based on hybrid conditions includes, in step S300: training a generative adversarial network model using a Wasserstein GAN framework with gradient penalty. The loss function of the discriminator includes adversarial loss, gradient penalty loss, and auxiliary classification loss, while the loss function of the generator includes adversarial loss and auxiliary classification loss.

[0056] Specifically, the training includes: using Wasserstein adversarial loss as the main loss and adding a gradient penalty term to the discriminator to constrain the 1-Lipschitz property of the discriminator function; simultaneously adding auxiliary classification cross-entropy loss to the losses of the generator and discriminator to strengthen class control; during training, the discriminator can be updated n_critic times (e.g., 1–5) in each round of updates, using Adam or its variants as the optimizer, and the learning rate and momentum parameters can be limited to the preferred range in the subordinate weights; the training termination condition is reaching the preset maximum number of iterations M or satisfying the loss stability criterion.

[0057] It is understandable that the loss weight, learning rate, and termination criterion can be set to different values ​​in different embodiments to balance training stability and generation quality; spectral domain reconstruction loss or MMD loss can also be added during training as an option to enhance distribution consistency.

[0058] S400: Uses a trained generator to generate synthetic hyperspectral samples based on specified mold level labels and random noise.

[0059] In this embodiment of the application, the hyperspectral data enhancement method based on mixed conditions includes step S400: applying a trained generator to generate a synthetic hyperspectral sample based on a specified mold grade label and random noise.

[0060] Specifically, the generation process includes: inputting the category label of the target mold level into the embedding layer and concatenating it with a random noise vector as the generator input; obtaining the style vector w through the Mapping Network; and the generator synthesizing hyperspectral samples step by step through the MSCA module and AdaIN transformation according to the hierarchy. Post-processing steps (such as inverse normalization, spectral smoothing, or screening based on discriminator confidence) can be applied to the generated original spectra. The quality and confidence of the generated samples can be automatically detected and evaluated through indicators such as reconstruction error, PCC, or spectral angle mapping (SAM).

[0061] It is understandable that random noise distribution, embedding dimension, post-processing threshold, and quality screening criteria can all be listed as implementation variations in the specification; in practical applications, rejection sampling or screening strategies can be used to retain only high-confidence synthetic samples into the enhancement set.

[0062] S500: Combines the synthesized hyperspectral samples with the original samples according to a preset mixing strategy to form an enhanced dataset.

[0063] In this embodiment of the application, the hyperspectral data enhancement method based on mixing conditions includes step S500: combining the synthesized hyperspectral sample with the original sample according to a preset mixing strategy to form an enhanced dataset.

[0064] Specifically, the preset mixing strategy includes, but is not limited to: fixed ratio mixing (e.g., synthetic:real = 0.1:1 to 2:1), dynamic compensation by class (adding synthetic samples to classes with fewer samples to balance the class distribution), screening based on confidence threshold (only including synthetic samples with discriminant or classifier confidence higher than τ), and removing abnormal spectra through data cleaning; after merging, the augmented dataset can be randomly shuffled again and divided according to the training / validation / test ratio for downstream training.

[0065] It is understood that the hybrid strategy can be flexibly selected based on downstream task preferences and resource constraints, and several specific hybrid ratios and confidence thresholds can be defined in the subordinate rights as preferred embodiments.

[0066] S600: Use augmented datasets to train a mold grade classification model.

[0067] In this embodiment of the application, the hyperspectral data augmentation method based on mixed conditions includes step S600: training a mold grade classification model using an augmented dataset.

[0068] Specifically, the training includes: selecting a classification network suitable for hyperspectral spectral sequences (e.g., 1D-CNN, 1D-ResNet, or a lightweight attention-based network) as the mold grade classification model, training the model using an augmented dataset, and using cross-validation, repeated experiments, and recording model performance metrics (e.g., overall accuracy OA, average accuracy AA, Kappa coefficient, and class recall / precision) during training to evaluate the improvement of classification performance by the augmented data; preferably, early stopping and regularization are implemented during the training process to avoid overfitting.

[0069] It is understandable that the specific architecture, hyperparameters, number of training epochs, and evaluation metrics of the classification model may vary in different implementations; in addition, the use of synthetic samples may be limited to the training phase and not used for testing, in order to truly evaluate generalization ability.

[0070] S700: Classify tobacco leaves according to the mold grade classification model.

[0071] In this embodiment of the application, the hyperspectral data enhancement method based on mixed conditions includes, in step S700: classifying tobacco leaves according to a mold grade classification model.

[0072] Specifically, the classification includes: in the inference stage, the tobacco leaf samples to be tested are collected and processed through the same preprocessing procedure as in the training stage to obtain the input spectral vector, which is then fed into the trained mold grade classification model to obtain the predicted category and confidence level; based on the prediction results, subsequent processing procedures can be triggered, such as automatic sorting, manual review or alarm prompts, and low-confidence samples can be fed back to the labeling process to gradually expand the real sample library.

[0073] It is understood that the classification can be deployed in the cloud or on edge devices, and the prediction threshold, decision rules and subsequent actions can be configured according to the application scenario (such as on-site real-time sorting or batch offline detection); it can also be combined with multi-temporal or multi-view data to further improve the robustness of the judgment.

[0074] In this embodiment, the hybrid conditional embedding mechanism further includes concatenating a random noise vector with the embedding vector of the mold grade category label as the initial input to the generator, thereby providing global category information guidance for the generative adversarial network model. A style mapping network maps the mold grade category label to an intermediate style vector to achieve hierarchical style modulation. The style mapping network is a multilayer perceptron used to convert the mold grade category label into an intermediate style vector, which is then used to modulate the feature map at each layer of the generator.

[0075] Specifically, the hybrid conditional embedding mechanism also includes concatenating a random noise vector with the embedding vector of the mold grade category label as the initial input to the generator, providing global category information guidance for the generative adversarial network model. A style mapping network maps the mold grade category label to intermediate style vectors to achieve hierarchical style modulation. This style mapping network is a multilayer perceptron used to convert the mold grade category label into intermediate style vectors, which are then used to modulate the feature map at each layer of the generator.

[0076] Specifically, the random noise vector z follows a standard normal distribution or a uniform distribution (e.g., z ~ N(0,1) or U(-1,1)). The mold level category label is mapped to a fixed-dimensional category embedding vector e (e.g., dimension 16–256) through an embedding layer. The initial input of the generator is formed by concatenating z and e by channel to form a vector [z; e]. This vector is expanded by several fully connected layers or linear transformations and used as the input tensor of the first layer of the generator. The style mapping network takes category embedding or category one-hot encoding as input and maps it to an intermediate style vector w through at least two fully connected layers and nonlinear activation. The intermediate style vector w generates the scaling factor γ(w) and bias β(w) required for each generation layer through linear layers or affine mapping, and is used in the AdaIN unit of each generation layer to perform affine modulation on the feature map of that layer by channel to achieve hierarchical style control and category condition guidance.

[0077] It is understandable that the noise distribution, embedding dimension, number of layers and hidden units of the Mapping Network, and generation methods of γ(w) and β(w) can be equivalently modified in different embodiments; the category embedding can adopt learned embedding, one-hot representation, or composite vector combining spectral statistical features; to improve numerical stability, a small constant ε can be added to the denominator in the AdaIN calculation to avoid division by zero, and dropout, LayerNorm, or residual connections can be added to the Mapping Network to improve training stability.

[0078] In this embodiment, the style mapping network is a multilayer perceptron, which contains at least two fully connected layers. In each processing block of the generator, the adaptive instance normalization layer modulates the feature map using a modulation formula:

[0079] in For the first Feature map of the layer and These represent the channel mean and standard deviation of the feature map, respectively. and To obtain from style vectors Derived scaling factor and bias factor.

[0080] Specifically, and Statistical calculations were performed for each channel of the feature map in the spectral / spatial dimensions (i.e., the mean and standard deviation of all spectral points or spatial locations in that channel were calculated). and From style vectors It is generated through one or more linear mappings (e.g., a single-layer linear transformation or a small fully connected subnetwork), and its output dimension is consistent with the number of channels in the i-th layer; in implementation, to avoid numerical instability, A small constant ε (e.g., 1e-5) is added to the calculation; AdaIN modulation applies scaling and translation along the channel dimension, so that the calculation results in a result that is... The determined category style information is injected into the generation process layer by layer.

[0081] Understandably, AdaIN can be replaced with functionally equivalent normalization and affine modules (such as InstanceNorm or LayerNorm with affine parameters). and The generation parameters can be shared or independent for each layer; meanwhile, the number of layers, activation functions (ReLU, LeakyReLU, GELU, etc.) and output dimensions of the style mapping network can be listed as preferred ranges in the specification to expand the scope of protection.

[0082] In the embodiments of this application, the loss function of the discriminator includes the discriminator adversarial loss, the gradient penalty loss, and the generator adversarial loss.

[0083] The expression for the discriminant's adversarial loss is as follows:

[0084] in To generate samples, For real samples, , These are the scores given by the discriminator's adversarial head to generated and real samples, respectively. The data distribution generated by the generator For the true data distribution, For category labels, For counter-loss.

[0085] The expression for gradient penalty loss is:

[0086] in, These are the interpolation points between the real sample and the generated sample. For the distribution of interpolation points, The discriminator's adversarial head output is about the interpolation point. gradient, It is the L2 norm. The loss is a gradient penalty.

[0087] The expression for generator adversarial loss is:

[0088] in, To combat loss in generators.

[0089] Specifically, the expected value E[·] can be estimated from mini-batch samples, the weight of the gradient penalty term is the training hyperparameter (e.g., in the range of 0.1–10), and the discriminator can be updated n_critic times in advance during training to stabilize the adversarial training; the auxiliary classification loss can be added to the generator and discriminator in the form of cross-entropy.

[0090] Understandably, the specific weights of the loss terms and the optimizer hyperparameters can be set through multiple sets of experiments to achieve the best training stability; the gradient penalty can be implemented using interpolation-based WGAN-GP or an alternative method based on spectral normalization, and the auxiliary classification loss can use variants such as label smoothing and focal loss to adapt to class imbalance.

[0091] In this embodiment, the multi-scale channel attention module includes three parallel depthwise separable convolutional branches. Each depthwise separable convolutional branch consists of depthwise convolutions and pointwise convolutions. The multi-scale channel attention module concatenates the feature maps output by each depthwise separable convolutional branch along the channel dimension, and then performs channel attention weighting through a compression and activation module. The compression and activation module includes a global average pooling layer and two fully connected layers to calculate the attention weights for each feature channel.

[0092] Specifically, the three parallel depthwise separable convolutional branches can use different kernel sizes k_1, k_2, k_3 (e.g., k_1=3, k_2=5, k_3=7, or generally taken from the set of integers 3–15) to cover different receptive fields. Each branch first performs channel-wise depthwise convolution (only along the spectral axis or spectral × spatial axis) on the input feature h, and then performs pointwise convolution to mix channel information to obtain the branch output. The branch outputs are concatenated in the channel dimension to form a multi-scale feature representation u. Global average pooling s is performed on u, and then it is passed through two fully connected layers (the first layer reduces the dimension, with a dimension reduction ratio r of 4–16; the second layer increases the dimension) and then activated by ReLU and sigmoid to obtain the channel attention weights α. Finally, the concatenated features are weighted by channel-wise multiplication (α⊙u) and then fed into the AdaIN layer after adjusting the number of channels by 1×1 convolution or linear transformation.

[0093] It is understandable that the number of branches, kernel size, dimensionality reduction ratio r, whether residual connections or inter-layer normalization are added, etc., can all be listed as optional embodiments; the channel attention structure can be replaced with CBAM, self-attention, or Transformer-based modules to obtain equivalent or improved effects; to save computation, the number of channels after pointwise convolution can be compressed before splicing.

[0094] In this embodiment of the application, the operational expression for the depthwise separable convolution branch in the multi-scale channel attention module is:

[0095] in For the input feature map, The kernel size is Depth convolution, This is a point-to-point convolution.

[0096] Specifically, DWConv and PWConv can use convolution kernels with or without bias terms, and can be combined with batch normalization / layer normalization and activation functions (such as ReLU or LeakyReLU) to improve expressive power. In implementation, to control the number of parameters, channel compression / expansion layers can be introduced before and after PWConv to adjust computational complexity and representation capacity.

[0097] It is understandable that the above operators and steps can be extended to any number of parallel branches (≥2), and the hyperparameters of DWConv such as convolution dimension, stride, and padding can be adjusted according to the input spectral length L and the target feature resolution; the channel reshaping after Concat can be achieved by pointwise convolution or 1×1 transformation to restore the required channel dimension.

[0098] The formula for concatenating the output feature maps of each branch along the channel dimension is:

[0099] in, These are the output feature maps of the three depthwise separable convolutional branches. This means that these feature maps are spliced ​​together along the channel dimension to form a multi-scale feature representation.

[0100] The formula for global average pooling is:

[0101] Where u is the input feature map and L is the spectral length.

[0102] Specifically, global average pooling independently calculates the average value for each channel, returning a channel description vector s of shape [C×1]. In the one-dimensional spectral sequence scenario, L is the number of spectral points. The pooling result is used as a compressed representation and input to two fully connected layers to calculate channel attention.

[0103] Understandably, pooling operations can be replaced by global max pooling, a combination of global average and max pooling, or weighted pooling to more richly represent channel information. For inputs with spatial dimensions, spectral axis pooling can be performed first, followed by spatial axis pooling, or two-dimensional pooling can be used to adapt to different input forms.

[0104] The formula for calculating channel attention weights is:

[0105] in and For trainable weight matrix, It is the sigmoid activation function. This is the activation function for the linear rectifier unit.

[0106] Specifically, The dimension is (C / r)×C. The dimension of s is C×(C / r), where r is the channel compression ratio (e.g., 4, 8 or 16). The calculation process is to first perform a dimension reduction mapping on s and then pass it through ReLU, then perform a dimension increase mapping and pass it through sigmoid to obtain the scaling factor α for each channel. The value of α is in the range of (0,1), and it can be multiplied with the original multi-scale features channel by channel to achieve channel weighting.

[0107] Understandable and The specific dimensions, compression ratio r, activation function selection, and whether or not a bias term is added can all be listed as preferred embodiments; if non-binary weights need to be output, the following can be used: Replace with softmax or sigmoid with a temperature coefficient to change the attention distribution characteristics.

[0108] The formula for weighted processing is:

[0109] in Represents the Hadamard product. The output feature map after attention weighting. This is the concatenated multi-scale feature representation. The output feature map of the multi-scale channel attention module is passed to the next layer of the network through an adaptive instance normalization layer.

[0110] Specifically, α is broadcast along the channel dimension to a tensor of the same shape as u and multiplied element-wise to obtain the weighted features. Subsequently, channel number recovery and nonlinear mapping can be performed through pointwise convolution or convolutional blocks, and then fed into AdaIN for affine modulation with style vector w; in the implementation, the channel number can be recovered and nonlinearly mapped, and then fed into AdaIN for affine modulation. Perform batch or instance normalization to stabilize training.

[0111] Understandably, weighted averages can be used to... Optional residual connections can be implemented to preserve the original input information, or regularization operations such as Dropout and LayerNorm can be inserted before and after weighting. If the number of channels of u and α do not match, the dimensionality can be adjusted first using linear transformation. The entire module can be deployed in parallel with multiple instances to enhance the expressive power with multiple feature streams.

[0112] In this embodiment of the application, the preset hybridization strategy includes adding samples whose confidence level of the classifier or discriminator for the synthetic samples is higher than a preset value to the augmented dataset.

[0113] Specifically, for each synthetic sample generated by the generator, the sample confidence is calculated based on the output probability / score of the trained mold grade classification model or discriminator. The synthetic sample is incorporated into the augmented dataset only when the confidence is not lower than a preset threshold τ. The confidence can be the maximum Softmax probability of the classifier for the sample, the normalized score of the adversarial head of the discriminator, or a weighted combination of the two. The threshold can be set as a globally fixed value or set intra-class thresholds for each category to control the admission of synthetic samples of different mold grades. Before being incorporated into the augmented dataset, the synthetic samples can be further screened by quality metrics such as spectral angle mapping (SAM), Pearson correlation coefficient (PCC), or reconstruction error, and low-quality samples are removed or downweighted. After merging, the augmented dataset can be deduplicated, balanced, and randomly shuffled, and sample weights can be assigned to the synthetic samples to reflect their confidence differences in downstream training.

[0114] It is understandable that the specific values ​​of the confidence threshold, intra-class threshold, weighted combination coefficient, and quality metric mentioned above can be adjusted according to the actual data distribution and task requirements. For example, τ can be taken in the range of 0.7–0.99. The confidence calculation can use uncertainty estimation methods such as single model or model ensemble / Monte Carlo dropout to improve the reliability of the judgment. The quality screening indicators can be combined or replaced with other spectral domain evaluation indicators as needed. In addition, the hybrid strategy can adopt iterative enhancement, that is, gradually expanding the enhancement set in several rounds and updating the confidence judgment rule with the latest trained classification model after each round to achieve adaptive sample compensation and model robustness improvement.

[0115] Please also refer to Table 1, which compares the parameters of different algorithms.

[0116]

[0117] Table 1: Comparison of parameters for different algorithms Here, KL represents Kullback-Leibler divergence, used to assess the similarity in statistical distribution between generated and real spectra. It represents the information loss of one distribution relative to another; a smaller value is better, indicating that the distribution of generated data is closer to that of real data. JS represents Jensen-Shannon divergence, similar to KL divergence, used to assess the alignment between the distributions of generated and real data; a smaller value is better, indicating that the two distributions are more similar. PCC represents Pearson correlation coefficient, used to assess the structural similarity between generated and real spectra; a larger value is better (range [-1, 1]), and a value closer to 1 indicates a stronger linear correlation. MMD represents Maximum Mean Discrepancy, used to assess the distributional distance between generated and real data in the feature space. In the Reproducing Kernel Hilbert Space (RKHS), it measures the difference between two probability distributions; a smaller value is better, indicating that the two distributions are closer.

[0118] As shown in Table 1, this embodiment designed a comprehensive comparative experiment to compare the performance of the proposed method with several representative GAN architectures. The experiment selected standard GAN based on MLP backbone, conditional GAN ​​(cGAN), conditional Wasserstein GAN with gradient penalty (cWGAN-GP), and DCGAN as baseline models. Experimental results show that, using the standard GAN as a baseline, its average KL divergence is 0.0263, JS divergence is 0.0729, MMD is 0.1662, and PCC is 0.9832. The cGAN, which introduces class labels, improves on all indicators, with an average PCC of 0.9910 and KL, JS, and MMD decreasing to 0.0259, 0.0699, and 0.1571, respectively, demonstrating that class guidance has a significant effect on improving the quality of spectral generation. cWGAN-GP further benefits from the gradient penalty mechanism, performing better in distribution alignment, with MMD decreasing to 0.1399, and the in-class indicators are more stable. The DCGAN has an MMD of 0.1328, slightly better than other baselines, but its KL and JS divergence values ​​are relatively high, mainly due to the fact that the CNN structure with fixed convolutional kernels cannot fully represent the multi-scale complex features of hyperspectral data.

[0119] The MSC-StyleACGAN method proposed in this application significantly outperforms the aforementioned traditional and conditional GAN ​​variants across all evaluation metrics. Its average KL divergence is 0.0237, JS divergence is 0.0684, and MMD is 0.0488, all the lowest values, while its PCC reaches 0.9934, the highest value. This performance improvement is mainly attributed to the synergistic effect of multi-scale feature extraction and hybrid conditionalization strategies, enabling the generator to simultaneously capture both coarse-grained and fine-grained spectral structures. Of particular note is that the performance variance of the method of this invention is extremely small across the three mold categories 0, 1, and 2, demonstrating excellent generalization and robustness. All three categories exhibit high PCC and low dispersion. Specifically, the PCC for severe mold (level 0) is 0.992 and the KL is 0.0273; the PCC for mild mold (level 1) is as high as 0.9951 and the KL is only 0.0220; and the PCC for normal samples (level 2) is 0.993 and the KL is 0.0219. This fully demonstrates that the method of this invention can accurately characterize subtle spectral features with class discrimination, effectively solving a key challenge in hyperspectral data enhancement.

[0120] Please refer to the following: Figures 2a to 2f , Figure 2a The principal component analysis visualization results for the severe mold category provided in the embodiments of this application. Figure 2b The principal component analysis visualization results for the mild mold category provided in the embodiments of this application. Figure 2c The principal component analysis visualization results for the normal category provided in the embodiments of this application. Figure 2d Visualization results of t-distribution random neighborhood embedding for severe mold categories provided in the embodiments of this application. Figure 2e Visualization results of t-distribution random neighborhood embedding for mild mold categories provided in embodiments of this application. Figure 2f Visualization results of t-distributed random neighborhood embeddings for normal categories provided in embodiments of this application.

[0121] Please refer to the embodiments in this application as well. Figures 2a to 2f By employing two dimensionality reduction methods—principal component analysis (PCA) and t-distributed random neighborhood embedding (t-SNE)—the high-dimensional real hyperspectral samples and the synthetic samples generated by the method of this invention are projected into a two-dimensional space for visualization and comparison. In the figure, green dots represent real samples, and yellow dots represent generated samples. The visualization results clearly show that for the three categories of severe mold, mild mold, and normal, the distribution of the generated samples exhibits significant spatial overlap with the distribution of the real samples.

[0122] Specifically, Figures 2a to 2c The PCA visualization results validated that the generated data and the real data have a high degree of alignment in the global variance structure; while Figures 2d to 2fThe t-SNE visualization results further demonstrate that the generated samples maintain good consistency with the real samples in terms of local neighborhood relationships. This consistent overlap observed in two different dimensionality reduction spaces strongly proves that the MSC-StyleACGAN model proposed in this application can effectively learn and capture the underlying manifold structure of the original hyperspectral data, and successfully preserve key class discrimination information during the generation process.

[0123] Understandably, the visualization results above confirm that the present invention can simultaneously and accurately characterize the differences between different mold grades (inter-class variation) and the diversity within the same grade (intra-class variation). This ability to generate high-quality, category-controllable samples ensures that the generated synthetic data not only has high fidelity but also provides high-quality training samples rich in discriminative information for downstream tobacco mold grade classification models, thereby directly contributing to a substantial improvement in the performance of the classification models.

[0124] Figure 3a The average formation spectrum of a severe mold category provided in one embodiment of this application. Figure 3b The average formation spectrum of a mild mold category provided in one embodiment of this application. Figure 3c The average generated spectrum of the normal category is provided in one embodiment of this application.

[0125] In the embodiments of this application, please refer to Figures 3a to 3c The figure illustrates the average spectral curves of samples generated from three categories—severe mold, mild mold, and normal—at different training periods (Epoch 1, 50, 500, 1000, 1500) during the training process of the method of this invention, and compares them with the average spectrum of the original real samples. This figure visually reveals the dynamic evolution of the generator's learning of the spectral distribution of real data.

[0126] Specifically, in the early stages of training (e.g., Epoch 50), the generated average spectra deviate significantly from the real spectra in key spectral ranges (e.g., 1180–1400 nm), indicating that the model has not yet fully learned class-specific spectral patterns. As training continues, the generated spectra gradually converge to the real spectra, achieving a high degree of structural alignment with the real spectra by Epoch 1000. Finally, after training stabilizes (Epoch 1500), the generated average spectra for all three classes are highly consistent with the real spectra.

[0127] Understandably, this evolutionary trend from bias and convergence to a final high degree of agreement strongly demonstrates the effectiveness and stability of the hybrid conditional embedding mechanism, multi-scale attention module, and WGAN-GP training framework employed in this invention. It shows that the generator can not only train stably but also gradually learn to accurately replicate the complex and subtle spectral features under different mold levels, thus ensuring the high fidelity of the final generated data. This is crucial for effectively improving the performance of downstream classification models using synthetic data.

[0128] Please also refer to Table 2, which is a comparison table of classification performance under different ratios of real and synthetic samples.

[0129]

[0130] In this embodiment, to quantitatively evaluate the actual effect of the proposed method in hyperspectral mold classification data augmentation, a one-dimensional convolutional neural network was used as the baseline classifier, and systematic validation was performed on an independent test set containing 3000 labeled samples. By setting different ratios of real and synthetic samples, including using only real samples (1:0), using only synthetic samples (0:1), and different mixed ratios, the classification performance was comprehensively evaluated from three dimensions: overall accuracy (OA), average accuracy (AA), and Kappa coefficient.

[0131] Specifically, the experimental data in Table 2 show that when the synthetic samples generated by this invention are mixed with real samples in a 1:1 ratio, the classifier achieves optimal performance on the test set, with an overall accuracy of 0.9565, an average accuracy of 0.9560, and a Kappa coefficient of 0.9347. These figures represent significant improvements compared to the baseline model trained using only real samples (OA: 0.9484, AA: 0.9475, Kappa: 0.9225). Particularly noteworthy is the significant improvement in classification accuracy for minority classes (such as the mild mold category), increasing from 0.9072 to 0.9333, demonstrating that this invention effectively alleviates the class imbalance problem.

[0132] Understandably, the quantitative evaluation results of this system fully validate that synthetic spectral data based on MSC-StyleACGAN can effectively enhance the performance of hyperspectral classification models. Performance comparisons under different mixing ratios show that appropriately introducing high-quality synthetic samples can enrich the diversity and balance of training data, thereby significantly improving the classifier's generalization ability on unknown samples. This confirms that this invention, as a hyperspectral data augmentation strategy, can provide reliable technical support for downstream tobacco mold grade classification tasks.

[0133] Figure 4This is a hyperspectral data enhancement system 10 based on mixed conditions provided in one embodiment of this application. For example... Figure 4 As shown, the hyperspectral data augmentation system 10 includes at least the following components: a data acquisition module 11, a network construction module 12, a network training module 13, and a classification module 14.

[0134] In this embodiment, the data acquisition module 11 is used to acquire the original hyperspectral dataset, which includes mold grade labels. Please refer to the following for details. Figure 1 The details of sections 3 to 3 and their corresponding descriptions are not repeated here.

[0135] In this embodiment, the network construction module 12 is used to construct a generative adversarial network (GAN) model, which includes a generator and a discriminator. The generator employs a hybrid conditional embedding mechanism and a multi-scale channel attention module. The hybrid conditional embedding mechanism includes: concatenating a random noise vector with a mold grade embedding vector as global category information guidance during the initial input stage; simultaneously, mapping category labels to intermediate style vectors through a style mapping network and performing hierarchical style modulation at each layer of the generator using an adaptive instance normalization layer. The multi-scale channel attention module includes multiple parallel depthwise separable convolutional branches, each with a different kernel size. The feature maps output by the depthwise separable convolutional branches are concatenated along the channel dimension and then weighted by the channel attention module. The discriminator uses an auxiliary classifier architecture, including an adversarial head for judging the authenticity of data and a classification head for predicting mold grades. For details, please refer to [the relevant documentation / reference]. Figure 1 The details of sections 3 to 3 and their corresponding descriptions are not repeated here.

[0136] In this embodiment, the network training module 13 is used to train a generative adversarial network model using the Wasserstein GAN framework with gradient penalty. The discriminator's loss function includes adversarial loss, gradient penalty loss, and auxiliary classification loss, while the generator's loss function includes adversarial loss and auxiliary classification loss. The trained generator is used to generate synthetic hyperspectral samples based on specified mold grade labels and random noise. The synthetic hyperspectral samples are then combined with the original samples according to a preset mixing strategy to form an augmented dataset. The augmented dataset is then used to train a mold grade classification model. For details, please refer to the attached documentation. Figure 1 The details of sections 3 to 3 and their corresponding descriptions are not repeated here.

[0137] In this embodiment, the classification module 14 is used to classify tobacco leaves according to a mold grade classification model. Please refer to the following for details. Figure 1 The details of sections 3 to 3 and their corresponding descriptions are not repeated here.

[0138] The hyperspectral data augmentation system 10 based on hybrid conditions provided in this application achieves fully automated processing of hyperspectral data augmentation through the coordinated operation of its various modules. Specifically, the data acquisition module 11 is responsible for collecting and preprocessing raw hyperspectral data, providing high-quality input for subsequent processing; the network construction module 12 constructs a generator with a hybrid conditional embedding mechanism and a multi-scale channel attention module, as well as a discriminator using an auxiliary classifier architecture. This innovative structure ensures the spectral fidelity and category accuracy of the generated data; the network training module 13 uses the Wasserstein GAN framework with gradient penalty for model optimization, and a carefully designed hybrid loss function ensures the stability and convergence of training; the classification module 14 uses the augmented dataset to train a mold grade classification model and achieves accurate classification of tobacco leaves. Through the organic coordination of its modules, the entire system effectively solves the problems of hyperspectral data scarcity and class imbalance, significantly improving the accuracy and reliability of tobacco mold detection.

[0139] Figure 5 This is an electronic device 20 provided in one embodiment of this application. For example... Figure 5 As shown, the electronic device 20 includes at least the following components: a processor 21 and a memory 22.

[0140] In this embodiment, the memory 22 is used to store executable instructions of the processor 21, which, when configured to execute instructions, implement... Figure 1 The method for hyperspectral data enhancement based on mixed conditions is shown.

[0141] In one embodiment of this application, the program operating in the electronic device 20 may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). The information processed by these devices is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (Flash ROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0142] It should be noted that a portion of the electronic device 20 described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0143] It should be noted that the term "computer" as used here refers to a computer built into electronic device 20, employing hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard drives built into the computer.

[0144] Furthermore, a "computer-readable recording medium" can include: a medium that dynamically stores a program for a short period of time, such as a communication line used when transmitting a program via a network such as the Internet or a communication line such as a telephone line; or a medium that stores a program for a fixed period of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining with programs already recorded in the computer.

[0145] Furthermore, the electronic device 20 in the above embodiments can also be implemented as an assembly (device group) composed of multiple devices. Each device constituting the device group can possess some or all of the functions or functional blocks of the electronic device 20 in the above embodiments. As a device group, it is sufficient to have all the functions or functional blocks of the electronic device 20.

[0146] It is understood that the hyperspectral data augmentation method, system, electronic device and storage medium based on mixed conditions provided in this application combine global category guidance with hierarchical style modulation through the mixed condition embedding mechanism in the generator, and effectively capture the multi-scale spectral features of hyperspectral data with the help of the multi-scale channel attention module. At the same time, it adopts a discriminator with an auxiliary classifier and the WGAN-GP training framework, which can generate synthetic hyperspectral data with high fidelity and distinct category features. This significantly alleviates the problem of insufficient model training caused by sample scarcity and class imbalance, and effectively improves the accuracy, robustness and generalization ability of the downstream tobacco mold grade classification model.

[0147] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A hyperspectral data enhancement method based on mixed conditions, applied to the detection of mold growth in tobacco leaves, characterized in that, The method includes: Obtain the raw hyperspectral dataset, which includes mold level labels; Construct a generative adversarial network (GAN) model, which includes a generator and a discriminator; The generator employs a hybrid conditional embedding mechanism and a multi-scale channel attention module. The hybrid conditional embedding mechanism includes: concatenating a random noise vector with a mold level embedding vector as global category information guidance during the initial input stage; simultaneously, mapping category labels to intermediate style vectors via a style mapping network; and using an adaptive instance normalization layer to perform hierarchical style modulation at each layer of the generator. The multi-scale channel attention module includes multiple parallel depthwise separable convolutional branches, each with a different kernel size. The feature maps output by the depthwise separable convolutional branches are concatenated along the channel dimension and then weighted by the channel attention module. The discriminator uses an auxiliary classifier architecture, including an adversarial head for judging data authenticity and a classification head for predicting mold levels. The generative adversarial network model is trained using the Wasserstein GAN framework with gradient penalty, wherein the loss function of the discriminator includes adversarial loss, gradient penalty loss and auxiliary classification loss, and the loss function of the generator includes adversarial loss and auxiliary classification loss; The trained generator is used to generate synthetic hyperspectral samples based on specified mold level labels and random noise. The synthesized hyperspectral samples are combined with the original samples according to a preset mixing strategy to form an enhanced dataset; The model for classifying mold grades was trained using an augmented dataset. Tobacco leaves are classified according to the mold grade classification model.

2. The hyperspectral data enhancement method based on mixed conditions according to claim 1, characterized in that, The hybrid conditional embedding mechanism also includes: The random noise vector is concatenated with the embedding vector of the mold level category label as the initial input of the generator, so as to provide global category information guidance for the generative adversarial network model. The mold level category label is mapped to an intermediate style vector through a style mapping network to achieve hierarchical style modulation. The style mapping network is a multilayer perceptron, which is used to convert the mold level category label into the intermediate style vector. The intermediate style vector is used to modulate the feature map at each layer of the generator.

3. The hyperspectral data enhancement method based on mixed conditions according to claim 2, characterized in that, The style mapping network is a multilayer perceptron, which contains at least two fully connected layers. In each processing block of the generator, the adaptive instance normalization layer modulates the feature map using a modulation formula, which is: ； in For the first Feature map of the layer and These represent the channel mean and standard deviation of the feature map, respectively. and To obtain from style vectors Derived scaling factor and bias factor.

4. The hyperspectral data enhancement method based on mixed conditions according to claim 3, characterized in that, The loss function of the discriminator includes discriminator adversarial loss, gradient penalty loss, and generator adversarial loss; The expression for the adversarial loss of the discriminator is as follows: ； in To generate samples, For real samples, , These are the scores given by the discriminator's adversarial head to generated and real samples, respectively. The data distribution generated by the generator For the true data distribution, For category labels, For counter-attack losses; The expression for the gradient penalty loss is: ； in, These are the interpolation points between the real sample and the generated sample. For the distribution of interpolation points, The discriminator's adversarial head output is about the interpolation point. gradient, It is the L2 norm. Gradient penalty loss; The expression for the generator's adversarial loss is: ； in, To combat loss in generators.

5. The hyperspectral data enhancement method based on mixed conditions according to claim 4, characterized in that, The multi-scale channel attention module includes three parallel depthwise separable convolutional branches; Each of the depthwise separable convolutional branches consists of depthwise convolution and pointwise convolution; The multi-scale channel attention module concatenates the feature maps output by each depthwise separable convolutional branch along the channel dimension, and then performs channel attention weighting through the compression and activation module. The compression and activation module includes a global average pooling layer and two fully connected layers, used to calculate the attention weights for each feature channel.

6. The hyperspectral data enhancement method based on mixed conditions according to claim 5, characterized in that, In the multi-scale channel attention module, the operational expression for the depthwise separable convolution branch is: ； in For the input feature map, The kernel size is Depth convolution, Point-to-point convolution; The formula for concatenating the output feature maps of each branch along the channel dimension is: ； in, These are the output feature maps of the three depthwise separable convolutional branches. This means that these feature maps are concatenated along the channel dimension to form a multi-scale feature representation; The formula for the global average pooling operation is: ； Where u is the input feature map and L is the spectral length; The formula for calculating the channel attention weight is as follows: ； in and For trainable weight matrix, It is the sigmoid activation function. The activation function for the linear rectifier unit; The formula for the weighted processing is: ； in Represents the Hadamard product. The output feature map after attention weighting. This represents the multi-scale feature representation after splicing. The output feature map of the multi-scale channel attention module is passed to the next layer of the network through the adaptive instance normalization layer.

7. The hyperspectral data enhancement method based on mixed conditions according to claim 1, characterized in that, The preset hybridization strategy includes: The classifier or discriminator adds the synthetic samples to the augmented dataset if the confidence level of the synthetic samples is higher than a preset value.

8. A hyperspectral data enhancement system based on hybrid conditions, characterized in that, The system includes: The data acquisition module is used to acquire the raw hyperspectral dataset, which includes mold grade labels; A network construction module is used to construct a generative adversarial network (GAN) model, which includes a generator and a discriminator. The generator employs a hybrid conditional embedding mechanism and a multi-scale channel attention module. The hybrid conditional embedding mechanism includes: concatenating a random noise vector with a mold level embedding vector as global category information guidance during the initial input stage; simultaneously, mapping category labels to intermediate style vectors through a style mapping network; and using an adaptive instance normalization layer to perform hierarchical style modulation at each layer of the generator. The multi-scale channel attention module includes multiple parallel depthwise separable convolutional branches, each with a different kernel size. The feature maps output by the depthwise separable convolutional branches are concatenated along the channel dimension and then weighted by the channel attention module. The discriminator uses an auxiliary classifier architecture, including an adversarial head for judging data authenticity and a classification head for predicting mold levels. The network training module is used to train the generative adversarial network model using the Wasserstein GAN framework with gradient penalty. The discriminator's loss function includes adversarial loss, gradient penalty loss, and auxiliary classification loss, while the generator's loss function includes adversarial loss and auxiliary classification loss. The trained generator is used to generate synthetic hyperspectral samples based on specified mold grade labels and random noise. The synthetic hyperspectral samples are then combined with the original samples according to a preset mixing strategy to form an augmented dataset. The augmented dataset is then used to train a mold grade classification model. The classification module is used to classify tobacco leaves according to the mold grade classification model.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the hyperspectral data enhancement method based on hybrid conditions as described in any one of claims 1 to 7 when executing the instructions.

10. A computer-readable storage medium, characterized in that, The instructions include instructions that direct the device to perform the hyperspectral data enhancement method based on mixed conditions as described in any one of claims 1 to 7.

Images