TSFS sample generation method based on improved generative adversarial network

Abstract

The invention discloses a TSFS sample generation method based on an improved generative adversarial network. The TSFS sample generation method comprises the following steps: step 1, acquiring TSFS data of a sample by using a three-dimensional fluorescence spectrometer; step 2, constructing a CA-WGAN-GP optimization model with a convolution attention mechanism, and performing data enhancement on an original TSFS sample; step 3, in a model training stage, storing generated data once after epoch training for a certain number of times, and enabling a generated sample coverage to have higher diversity by utilizing generator models of a plurality of training stages; 4, extracting enhanced sample features by using a pre-trained CNN model, projecting the sample features by using linear discriminant analysis LDA, and preliminarily screening samples according to a Mahalanobis distance and a set threshold value; and 5, performing secondary screening by using a K-means algorithm to obtain a final training sample set. According to the method, the time-consuming and labor-consuming problem of TSFS sample acquisition is effectively solved, the model over-fitting phenomenon is reduced, and the robustness of unknown data is improved.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a TSFS sample generation method based on an improved generative adversarial network. Background Art

[0002] The TSFS technique needs to maintain a fixed interval (Δλ) between the excitation light and emission light wavelengths during the scanning process, which requires synchronous scanning over a wide wavelength range. In order to accurately capture the complex characteristics of the sample at multiple excitation and emission wavelengths, a smaller wavelength step is usually used for high-resolution scanning. This high-resolution synchronous scanning significantly increases the number of data collection points and extends the total experimental time. In addition, in order to improve the signal-to-noise ratio, a longer integration time must be performed at each wavelength position to enhance the signal intensity and obtain a better signal-to-noise ratio. These technical requirements make the TSFS sample acquisition process longer than traditional fluorescence spectroscopy techniques.

[0003] At the same time, TSFS technology aims to produce sharper and narrower spectra to effectively eliminate light scattering interference. This requires precise measurement and processing of fluorescence signals, which may involve multiple scans or signal averaging, further increasing the experimental time. The characteristics of the sample itself may also affect the data acquisition time: for samples with complex fluorescence characteristics or weak fluorescence intensity, longer measurement times are required to obtain sufficient signal intensity and resolution. Therefore, the combined influence of technical requirements and sample characteristics makes it usually take a long time to acquire TSFS samples. Summary of the invention

[0004] In response to the technical problem of time-consuming TSFS sample collection mentioned above, this technical solution provides a TSFS sample generation method based on an improved generative adversarial network. Through unsupervised learning, the mapping relationship between random noise and real sample training sets is mastered to generate high-quality TSFS samples, thereby effectively reducing the time and resources required to obtain TSFS samples; effectively solving the above problems.

[0005] The present invention is achieved through the following technical solutions:

[0006] A TSFS sample generation method based on an improved generative adversarial network, the method comprising the following steps:

[0007] Step 1: Use a three-dimensional fluorescence spectrometer to obtain sample TSFS data;

[0008] Step 2: Construct a CA-WGAN-GP optimization model with a convolutional attention mechanism to perform data augmentation on the original TSFS samples; the CA-WGAN-GP optimization model consists of a generator and a discriminator. The generator includes a linear layer, a convolutional transpose block, a self-attention mechanism, and a residual block; the discriminator includes a linear layer and a small batch discriminant layer;

[0009] Step 3: During the model training phase, after a certain number of training epochs, the generated data is stored once, and the generator model of multiple training phases is used to make the generated sample coverage more diverse;

[0010] Step 4: Use the pre-trained CNN model to extract the enhanced sample features, use linear discriminant analysis (LDA) to project the sample features, and preliminarily screen the samples based on the Mahalanobis distance and the set threshold;

[0011] Step 5: Use the K-means algorithm to perform secondary screening to obtain the final training sample set.

[0012] Furthermore, the TSFS data described in step 1 maintains a fixed interval Δλ between the wavelengths of the excitation light and the emission light during the scanning process.

[0013] Furthermore, the linear layer included in the generator in step 2 is used for feature mapping, the self-attention mechanism captures the long-range dependencies in the data, and the residual block helps generate more stable features through short-circuit connections; the convolution transpose block gradually increases the size of the feature map through reverse convolution, and introduces Gaussian noise to enhance the diversity and robustness of the samples; then, the standard convolution layer, ReLU activation function and batch normalization layer post-process the generated features, introduce nonlinearity and stabilize the training process.

[0014] Furthermore, when designing the generator described in step 2, the input is first constructed by embedding noise and labels. The noise vector is generated by standard normal distribution, and the label is converted into a vector of fixed dimension through the embedding layer, so that the generator can generate samples of a specific category according to the label; after the noise and label are embedded and concatenated, preliminary feature mapping is performed through the fully connected layer, and then the feature dimension is gradually enlarged by the transposed convolution layer so that it finally matches the real data dimension; and in order to prevent the generated samples from being too single or too noisy, a Gaussian noise layer is added after each convolution layer; at the same time, the residual block further enhances the feature expression ability of the generator, and the self-attention mechanism captures the global information of the input data to ensure that the generated samples have long-distance dependencies and generate more consistent and diverse samples; finally, the output of the generator is limited to a reasonable range through the tanh activation function.

[0015] Furthermore, the linear layer included in the discriminator in step 2 gradually reduces the data dimension and introduces nonlinearity through the Leaky ReLU activation function, so that the model can better capture complex features; the small batch discriminant layer is used to enhance the sensitivity of the discriminator to small batch samples and improve the ability to distinguish between generated samples and real samples.

[0016] Furthermore, the discriminator described in step 2 is designed to distinguish between fake samples and real samples generated by the generator; the label information is spliced ​​with the input data through the label embedding layer, so that the discriminator can process data and label information at the same time; in the fully connected network, nonlinear mapping is performed through multi-layer linear transformation and LeakyReLU activation function to gradually compress the features of the input data and output a scalar to represent the authenticity of the input data; at the same time, in order to prevent the generator from generating highly similar samples, a small batch discriminant layer is introduced to calculate the differences between samples in the batch to ensure that the discriminator can identify small differences between different generated samples; so that the discriminator can provide effective supervision signals to help the generator generate more diverse and realistic samples.

[0017] Furthermore, in the model training phase described in step 3, the generator and the discriminator are trained together through a competition mechanism. As the training progresses, after reaching a certain point, although the loss tends to be stable, the models at different times still have a certain degree of diversity, making it difficult to select the best model. Therefore, after a certain number of training epochs, the generated data is stored once, and the generator models of multiple training stages are used to generate samples with higher diversity, effectively improving the diversity and coverage of generated samples;

[0018] Furthermore, in the model training phase described in step 3, the TSFS data described in step 1 is divided into a training set, a validation set, and a test set, and only the training data is used for data augmentation; during the training and optimization process, the generator and the discriminator use different learning rates to achieve their respective optimal effects; the generator uses a higher learning rate to ensure that realistic samples are quickly generated in each update step, while the discriminator uses a lower learning rate to ensure stable judgment of fake samples; during training, the update frequency of the discriminator is much higher than that of the generator to maintain the dominant position of the discriminator, thereby Provide effective learning feedback for the Generator; in each training cycle, the loss of the Generator improves the generated samples by maximizing the misjudgment probability of the Discriminator, while the Discriminator improves its discrimination ability by maximizing the score of real samples and minimizing the score of generated samples; during the training process of the model, the losses of the Generator and the Discriminator are recorded to help analyze the training status of the model, and understand the game of generation and discrimination by drawing the loss curve; and after the loss curve stabilizes, the Generator saves the model once every certain training epoch, and adjusts the model hyperparameters through the actual output.

[0019] Furthermore, the Mahalanobis distance described in step 4 is used for the first screening of samples. The sample selection strategy in the CA-WGAN-GP optimization model first divides the original data set into a training set, a validation set, and a test set; the image data in the training set is feature extracted using a pre-trained CNN model (such as VGG16, ResNet50, etc.) and converted into a feature vector; the sample features are projected using linear discriminant analysis LDA; the samples of each category are preliminarily screened according to the Mahalanobis distance; by setting a threshold, the samples far away from the center of each category in the feature space of the training set are filtered out to ensure that the selected samples are more representative and reduce the potential negative impact of outliers on model training.

[0020] Furthermore, a pre-trained CNN model is used to extract deep features of each image, and then the mean vector and covariance matrix of the features are calculated according to the category, and the distance between each sample and its category center is measured by the Mahalanobis distance. In order to evaluate the distribution of samples, the model calculates the Mahalanobis distance of all samples in each category, and displays the sample distribution of each category through a visualization tool, intuitively showing samples with a large distance from the category center, which facilitates further sample screening and model optimization.

[0021] Furthermore, the specific operation method of step 5 is as follows:

[0022] Step 5.1: For the samples after preliminary screening, the K-means algorithm uses the same pre-trained CNN model as step 4 to extract features from the image data of the validation set;

[0023] Step 5.2: Apply the K-means algorithm to these feature vectors to divide each category into multiple clusters and determine the center of each cluster;

[0024] Step 5.3: Apply the K-means model trained on the validation set to the sample set after preliminary screening, and select the sample that best represents the center of each cluster from each category;

[0025] Step 5.4: The selected samples constitute the final training sample set.

[0026] Furthermore, the validation set and the test set always remain unchanged, the validation set is used for hyperparameter adjustment during the model training process, and the test set is used for the final model evaluation.

[0027] Beneficial Effects

[0028] The TSFS sample generation method based on an improved generative adversarial network proposed in the present invention has the following beneficial effects compared with the prior art:

[0029] (1) The present invention generates diversified TSFS samples through the CA-WGAN-GP model to effectively capture the spectral variability in real samples, while accurately extracting and learning the potential features of different spectral components, thereby improving the accuracy and robustness of spectral analysis. The CA-WGAN-GP model adopts adversarial training of the generator (G) and the discriminator (D). G generates diversified spectral samples through conditional labels and random noise, and is continuously optimized under the feedback of D. It can simulate the spectral variability in real samples, including complex phenomena such as spectral drift, overlap and nonlinear characteristics. The self-attention mechanism helps the generator to pay global attention to the correlation between different bands of the spectrum, capture potential spectral feature patterns, avoid local information loss, and improve the ability to analyze complex spectra. In addition, the small batch discrimination mechanism introduced by the model enhances the accuracy and representativeness of spectral data learning by identifying the feature differences between samples in the batch. At the same time, the Mahalanobis distance is combined to evaluate the distance between the sample and the category center to ensure the similarity between the generated sample and the real sample. To further improve the effectiveness of sample selection.

[0030] (2) The present invention also uses the K-Means clustering algorithm to select the most representative samples from the generated diverse samples, and by performing cluster analysis on the samples of each category, ensure that the selected samples can reflect the feature distribution within the category. The gradient penalty and spectral normalization mechanism in CA-WGAN-GP ensure the stability of model training, making the generated spectral data closer to the real distribution, and without the need for cumbersome manual feature extraction steps. This automated generation and screening method combining Mahalanobis distance and K-Means clustering not only improves the robustness and accuracy of spectral analysis, but also can adapt to different experimental conditions and application scenarios, providing an efficient, accurate and stable solution for spectral data analysis in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0032] Figure 2 This is the structural diagram of the CA-WGAN-GP model in the present invention.

[0033] Figure 3 This is a flow chart of sample screening in the present invention.

[0034] Figure 4 This is the flow chart of the K-means algorithm in the present invention.

[0035] Figure 5 These are real sample images of goat milk scanned using the same processing method during the five time periods in the embodiment.

[0036] Figure 6A sample graph is generated for CA-WGAN-GP in the embodiment.

[0037] Figure 7 : Comparison between a single real sample and a generated sample when Δλ=45 nm in the embodiment. DETAILED DESCRIPTION

[0038] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Under the premise of not departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solutions of the present invention should all fall within the protection scope of the present invention.

[0039] Embodiment 1:

[0040] A TSFS sample generation method based on an improved generative adversarial network. Aiming at the time-consuming problem of TSFS sample collection, the present invention first applies the CA-WGAN-GP model to expand the sample; then, uses the pre-trained CNN model to extract sample features, and performs preliminary screening based on the Mahalanobis distance and the set threshold; the secondary screening uses the K-means algorithm to obtain the final generated sample set of a specified number. The specific operation method is as follows Figure 1 As shown, the following steps are included:

[0041] Step 1: Use a 3D fluorescence spectrometer to obtain sample TSFS data, and keep a fixed interval Δλ between the excitation light and emission light wavelengths during the scanning process; the specific operation method is:

[0042] Obtain sample TSFS data. Taking the TSFS data collection of goat milk products as an example, raw goat milk comes from Yangyan Farm in Shangdang District, Changzhi City, Shanxi Province. After the milk samples are collected, they are pasteurized and immediately placed in a -30°C environment for low-temperature freezing. A total of 1500 ml of sterilized raw goat milk samples are collected. Then they are transported to the laboratory through a cold chain and refrigerated in a -20°C freezer, and the samples are kept at this temperature until the experiment. Before measurement, the raw goat milk samples are heated in a 20°C water bath for 1 hour, and then centrifuged at 5000r / min. -1The samples were centrifuged at a speed of 15 min to remove the upper fat layer, and the lower layer samples were shaken and TSFS data were collected using an LS55 fluorescence photometer. The photometer excitation wavelength range was set to 200nm-600nm, and two monochromators were scanned simultaneously at 0.5nm increments to obtain TSFS, Δλ was 30nm-195nm, the step length was 15nm, the slit width was 9nm, and the scanning speed was 1500nm / min. All processing steps were kept consistent, and the measurements were made 4 times a day for 5 consecutive days. A total of 20 parallel TSFS goat milk samples were obtained, and the size of each sample was (12, 801). In order to maximize the integrity of all spectral data, each spectral data sample was flattened into one-dimensional data (1, 9612) in sequence when data enhancement was performed.

[0043] Step 2: Build a CA-WGAN-GP optimization model with convolutional attention mechanism to perform data augmentation on the original TSFS samples; Figure 2 As shown, the CA-WGAN-GP optimization model is composed of a generator (G) and a discriminator (D).

[0044] The generator includes linear layers, convolutional transpose blocks, self-attention mechanisms, and residual blocks; the linear layers are used for feature mapping, the self-attention mechanism captures long-range dependencies in the data, and the residual blocks help generate more stable features through short-circuit connections; the convolutional transpose block gradually increases the size of the feature map through reverse convolution and introduces Gaussian noise to enhance the diversity and robustness of the samples; subsequently, the standard convolutional layer, ReLU activation function, and batch normalization layer post-process the generated features, introduce nonlinearity, and stabilize the training process.

[0045] When designing the generator, the input is first constructed by embedding noise and labels. The noise vector is generated through a standard normal distribution, and the label is converted into a vector of fixed dimension through the embedding layer, so that the generator can generate samples of a specific category according to the label; after the noise and label are embedded and concatenated, preliminary feature mapping is performed through the fully connected layer, and then the feature dimension is gradually enlarged by the transposed convolution layer so that it finally matches the real data dimension; and in order to prevent the generated samples from being too single or too noisy, a Gaussian noise layer is added after each convolution layer; at the same time, the residual block further enhances the feature expression ability of the generator, and the self-attention mechanism captures the global information of the input data to ensure that the generated samples have long-distance dependencies and generate more consistent and diverse samples; finally, the output of the generator is limited to a reasonable range through the tanh activation function.

[0046] The discriminator includes a linear layer and a mini-batch discriminant layer. The linear layer contained in the discriminator gradually reduces the data dimension and introduces nonlinearity through the Leaky ReLU activation function, so that the model can better capture complex features. The mini-batch discriminant layer is used to enhance the discriminator's sensitivity to mini-batch samples and improve its ability to distinguish between generated samples and real samples.

[0047] The discriminator is designed to distinguish between fake samples and real samples generated by the generator; the label information is spliced ​​together with the input data through the label embedding layer, so that the discriminator can process data and label information at the same time; in the fully connected network, nonlinear mapping is performed through multi-layer linear transformation and LeakyReLU activation function to gradually compress the features of the input data and output a scalar to represent the authenticity of the input data; at the same time, in order to prevent the generator from generating highly similar samples, a small batch discriminant layer is introduced to calculate the differences between samples in the batch to ensure that the discriminator can identify small differences between different generated samples; so that the discriminator can provide effective supervision signals to help the generator generate more diverse and realistic samples.

[0048] The generator first maps the input category label to a 50-dimensional embedding vector through an embedding layer. This label embedding is concatenated with a 100-dimensional random noise vector to form the input of the generator. The concatenated input passes through a fully connected layer to map it to a high-dimensional space of 128×75=9600 dimensions. This fully connected layer uses the ReLU activation function and uses Dropout to prevent overfitting, with the dropout probability set to 0.25. Next, G is upsampled through two layers of transposed convolution. The first layer keeps the input dimension at 128 and upsamples the size of the feature map from 75 to 150 through a transposed convolution with a kernel size of 4, a stride of 2, and a padding of 1. The second layer reduces the feature dimension from 128 to 64 and further expands the feature map size from 150 to 300. After each layer of convolution, BatchNorm1d is added for normalization, the momentum is set to 0.8 to stabilize the training process, and the nonlinear features are maintained through the ReLU activation function. At the same time, a Gaussian noise layer is added after each convolution layer, and the standard deviation of the noise is 0.1 to increase the diversity of the generated samples. G also introduces a self-attention mechanism to enable it to capture global dependencies. Specifically, the input 128-dimensional features are first generated by two convolution operations to generate Query (query vector) and Key (key vector). The Query of each feature point indicates that it is looking for other features related to itself, while the Key represents the information provided by the feature point. These two vectors are mapped to the 16-dimensional feature space respectively, and the similarity between them is calculated by the inner product to generate the similarity matrix between the feature points. Then, the similarity matrix is ​​converted into attention weights through Softmax normalization, which indicates the degree of attention of each feature point to other feature points. Finally, the input features are weighted and summed using Value (value vector), and the weighted features are mapped back to the original 128-dimensional feature space. This self-attention mechanism enables the generator to generate samples with global consistency and rich details. Next, G further processes the generated features through a residual block, which consists of two convolution operations, with input and output dimensions of 128, a kernel size of 3, a stride of 1, and a padding of 1 to keep the feature map size unchanged. The residual block introduces a skip connection to add the input and output to prevent the gradient from disappearing in the deep network, ensuring that key information is still maintained during deep processing. Finally, the generator maps the high-dimensional feature map to the same 9612-dimensional spectral data as the real data through a fully connected layer, and limits the generated sample values ​​to the range of [-1,1] through the Tanh activation function to make it consistent with the real data distribution.

[0049] The discriminator is responsible for distinguishing generated samples from real samples. First, the input category label is converted into a 9612-dimensional embedding vector through the embedding layer, and then concatenated with the spectral data to form a 19224-dimensional input. D consists of three fully connected layers. The first layer maps the 19224-dimensional input to 1024 dimensions, using the Leaky ReLU activation function with a negative slope of 0.2 to ensure that the negative gradient does not disappear. The second fully connected layer further compresses the 1024 dimension to 512 dimensions, and the third layer compresses the 512 dimension to 256 dimensions, both using the Leaky ReLU activation function to reduce the feature dimension layer by layer. In the 256-dimensional feature space, D introduces a small batch discrimination mechanism, which uses 50 kernels, each with a dimension of 5, to compare the feature differences between samples in the same batch to prevent G from generating highly similar samples. This small batch discrimination mechanism enhances D's sensitivity to the true and false classification of samples by capturing the feature differences of samples within the batch. Finally, D concatenates 256-dimensional + 50-dimensional features through a fully connected layer and maps them to a scalar, and the output represents the probability of the input sample being true or false. During the entire training process, the model introduces a gradient penalty mechanism to ensure the gradient stability of D. The gradient penalty calculates the gradient norm by interpolating between the real sample and the generated sample to ensure that the gradient size is close to 1, thereby avoiding the dynamic imbalance between G and D. G and D are optimized using the Adam optimizer, where the learning rate of G is 0.002, the learning rate of D is 0.0004, and the momentum parameters are 0.5 and 0.9, ensuring the stability and efficiency of the optimization process. During the training process, the losses of G and D will be recorded and visualized through the drawing tool to show the convergence of the model. At the same time, after the loss curve is stable, G saves the model every certain training epoch. The detailed parameter settings of the model are shown in Tables 1 and 2.

[0050] Table 1. CA-WGAN-GP generator related parameters

[0051]

[0052]

[0053] Table 2. CA-WGAN-GP discriminator related parameters

[0054]

[0055] Step 3: During the model training phase, after a certain number of training epochs, the generated data is stored once, and the generator model of multiple training phases is used to make the generated sample coverage more diverse;

[0056] During the model training phase, the generator and the discriminator are trained together through a competition mechanism. As the training progresses, after a certain point, although the loss tends to be stable, the models at different times still have a certain degree of diversity, making it difficult to select the best model. Therefore, after a certain number of training epochs, the generated data is stored once, and the generator models of multiple training stages are used to generate samples with higher diversity, effectively improving the diversity and coverage of generated samples.

[0057] In step 1, the original data set is divided into training set, validation set and test set, and only training data is used for data augmentation. During the training and optimization process, the generator and discriminator use different learning rates to achieve their respective optimal effects. The generator uses a higher learning rate to ensure that realistic samples are quickly generated in each update step, while the discriminator uses a lower learning rate to ensure stable judgment of fake samples. During training, the update frequency of the discriminator is much higher than that of the generator to maintain the dominant position of the discriminator, thereby providing the generator with a better performance. r provides effective learning feedback; in each training cycle, the loss of the generator improves the generated samples by maximizing the misjudgment probability of the discriminator, while the discriminator improves its discrimination ability by maximizing the score of the real samples and minimizing the score of the generated samples; during the training process of the model, the losses of the generator and the discriminator are recorded to help analyze the training status of the model, and understand the game of generation and discrimination by drawing the loss curve; and after the loss curve stabilizes, the generator saves the model once every certain training epoch, and adjusts the model hyperparameters through the actual output.

[0058] Step 4: Use the pre-trained CNN model to extract enhanced sample features, use linear discriminant analysis (LDA) to project the sample features, and preliminarily screen the samples based on the Mahalanobis distance and the set threshold.

[0059] A pre-trained CNN model is used to extract deep features of each image. The mean vector and covariance matrix of the features are then calculated according to the category. The distance between each sample and the center of its category is then measured using the Mahalanobis distance. In order to evaluate the distribution of samples, the model calculates the Mahalanobis distance of all samples in each category and displays the sample distribution of each category using a visualization tool, intuitively showing samples with a large distance from the category center, which facilitates further sample screening and model optimization.

[0060] Figure 3 The sample screening flow chart is shown in the figure. Since the CNN classifier performs better when processing image data, this study converts the TSFS spectral data into contour maps with a unified image size of 224×224 pixels. Taking the classic AlexNet network as an example, the pre-trained Alex Net is used to extract high-dimensional features of the image. The model was trained on the ImageNet dataset, so it already has a strong ability to extract image features. In order to focus on feature extraction, the last fully connected layer of the model is removed, and only the convolution and pooling layers are retained, through which the deep features of the input image are extracted. The model is loaded onto the GPU device and set to evaluation mode to avoid unnecessary gradient calculations. This evaluation mode disables the training behavior of certain layers in the model, thereby ensuring the stability of the model behavior during inference. After the dataset is loaded and the model is ready, the next step is to input the image data into the pre-trained AlexNet model to extract the features of each image. The output of the model is a high-dimensional feature vector. In order to facilitate subsequent calculations, the output is converted into a one-dimensional feature vector. The extracted image features are then processed by category. For each category, extract all sample features of the category, and calculate the mean vector of these features as the feature center of the category. Since the Mahalanobis distance not only takes into account the distance between the sample and the mean, but also the correlation between the features, it is necessary to calculate the covariance matrix of the category features. By using the pinv() function to calculate the generalized inverse matrix of the covariance matrix, the situation where the covariance matrix is ​​not invertible can be avoided. For each sample in each category, calculate its Mahalanobis distance to the category center. Specifically, the distance value of the sample is calculated by combining the mean vector and the inverse of the covariance matrix through the mahalanobis() function. The larger the distance value, the greater the deviation of the sample from the category center, and it may be more difficult to be correctly classified.

[0061] The Mahalanobis distance is used for the first screening of samples. The sample selection strategy in the CA-WGAN-GP optimization model first divides the original data set into training set, validation set and test set; the image data in the training set uses a pre-trained CNN model (such as VGG16, ResNet50, etc.) for feature extraction and converts it into a feature vector; the sample features are projected using linear discriminant analysis (LDA); the samples of each category are preliminarily screened according to the Mahalanobis distance; by setting a threshold, the samples that are far away from the center of each category in the feature space of the training set are filtered out to ensure that the selected samples are more representative and reduce the potential negative impact of outliers on model training.

[0062] The Mahalanobis distance is used to measure the distance between the sample and the mean of the category, while taking into account the correlation between features. Before calculating the Mahalanobis distance, it is necessary to construct a covariance matrix. For each category feature X, the covariance matrix calculation formula is as follows:

[0063]

[0064] Where Σ is the covariance matrix, X i is the feature vector of the i-th sample, μ is the mean vector of the category feature, and N is the number of samples in this category. Since the covariance matrix may be singular (i.e., irreversible), the present invention obtains the generalized inverse matrix through the Moore-Penrose inverse matrix:

[0065] Σ + =(Σ T Σ) -1 Σ T

[0066] For each category of features, the mean vector μ is calculated as follows:

[0067]

[0068] The definition of Mahalanobis distance is as follows:

[0069]

[0070] The Mahalanobis distance threshold is set according to the original samples to filter out samples far away from the center of each category in the training feature space to ensure that the selected samples are more representative and reduce the negative impact of outliers on model training.

[0071] Step 5: Use the K-means algorithm to perform secondary screening to obtain the final training sample set. K-means is an unsupervised learning algorithm that aims to divide data points into K clusters. The goal of the algorithm is to minimize the sum of the squares of the distances between each data point and its corresponding cluster center. The K-means algorithm process in sample screening is as follows: Figure 4 As shown, the specific operation method is:

[0072] Step 5.1: Use the K-means clustering method to select representative samples from each category. In order to ensure that each category can select samples that represent the category, for the samples after preliminary screening, the K-means algorithm uses the same pre-trained CNN model as step 4 to extract features from the image data of the validation set.

[0073] Step 5.2: Apply the K-means algorithm to these feature vectors, divide each category into multiple clusters, and determine the center of each cluster; specifically: Use the K-means algorithm to divide these features into several clusters, and the center point of each cluster represents a certain characteristic pattern of the category. By calculating the Euclidean distance between the center point of each cluster and the sample feature, select the sample closest to the cluster center as the representative sample. In each category, according to the predetermined sample quantity requirements, select the corresponding number of samples through the cluster center.

[0074] Step 5.3: Apply the K-means model trained on the validation set to the sample set after preliminary screening, and select the sample that best represents the center of each cluster from each category.

[0075] Step 5.4: The selected samples constitute the final training sample set.

[0076] In the above process, the validation set and test set remain unchanged. The validation set is used for hyperparameter adjustment during model training, and the test set is used for final model evaluation. The whole process uses the pre-trained model to extract high-dimensional features and combines the K-means clustering algorithm to select the most representative samples, so that the final data set can better reflect the characteristics of each category, while retaining important original samples to ensure the integrity and diversity of the data.

[0077] In the present invention, the optimization objective function of K-means is:

[0078]

[0079] Among them, J is the loss function used to measure the quality of clustering, K is the number of clusters, and N represents the total number of data points in a particular cluster. is the i-th sample belonging to the k-th cluster, μ k is the centroid of the kth cluster.

[0080] When calculating the distance between the sample feature and the cluster center, the norm between the feature vectors is used, and the formula is:

[0081]

[0082] Where x and y are the feature vectors of the two samples respectively. The order of the p-norm determines the type and nature of the norm. n is the dimension of vector x and vector y. i and i are the coordinates of vector x and vector y in the i-th dimension respectively.

[0083] Figure 5 The following are real samples of goat milk scanned in the same way over five time periods. Figure 6The sample graph generated by CA-WGAN-GP is shown. Figure 7 It shows the comparison between a single real sample and the generated sample when Δλ=45nm, where the red solid line represents the real sample and the black dotted line represents the generated sample.

[0084] The results show that the generated samples are very similar to the real samples of the corresponding categories. Although not completely consistent, they just meet the requirements that the generated samples need to be consistent with the distribution of real samples and have a certain diversity. With the help of the discriminator, the generator successfully learns the different characteristics of each category and can generate diversified samples according to the category. In the specific implementation of the present invention, the description not involved belongs to the known technology in the field and can be implemented with reference to the known technology.

[0085] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the present invention.

Claims

1. A TSFS sample generation method based on an improved generative adversarial network, characterized by: The method comprises the following steps: Step 1: Use a three-dimensional fluorescence spectrometer to obtain sample TSFS data; Step 2: Construct a CA-WGAN-GP optimization model with a convolutional attention mechanism to perform data augmentation on the original TSFS samples; the CA-WGAN-GP optimization model consists of a generator and a discriminator. The generator includes a linear layer, a convolutional transpose block, a self-attention mechanism, and a residual block; the discriminator includes a linear layer and a small batch discriminant layer; Step 3: During the model training phase, after a certain number of training epochs, the generated data is stored once, and the generator model of multiple training phases is used to make the generated sample coverage more diverse; Step 4: Use the pre-trained CNN model to extract the enhanced sample features, use linear discriminant analysis (LDA) to project the sample features, and preliminarily screen the samples based on the Mahalanobis distance and the set threshold; Step 5: Use the K-means algorithm to perform secondary screening to obtain the final training sample set.

2. The TSFS sample generation method based on an improved generative adversarial network according to claim 1, characterized in that: The TSFS data described in step 1 maintains a fixed interval between the excitation and emission wavelengths during the scan. .

3. The TSFS sample generation method based on an improved generative adversarial network according to claim 1, characterized in that: The linear layer contained in the generator described in step 2 is used for feature mapping, the self-attention mechanism captures long-range dependencies in the data, and the residual block helps generate more stable features through short-circuit connections; the convolutional transpose block gradually increases the size of the feature map through reverse convolution and introduces Gaussian noise to enhance the diversity and robustness of the samples; then, the standard convolutional layer, ReLU activation function and batch normalization layer post-process the generated features, introduce nonlinearity and stabilize the training process.

4. A TSFS sample generation method based on an improved generative adversarial network according to claim 1 or 3, characterized in that: When designing the generator described in step 2, the input is first constructed by embedding noise and labels. The noise vector is generated by standard normal distribution, and the label is converted into a vector of fixed dimension through the embedding layer, so that the generator can generate samples of specific categories according to the label; after the noise and label are embedded and spliced, preliminary feature mapping is performed through the fully connected layer, and then the feature dimension is gradually enlarged by the transposed convolution layer so that it finally matches the real data dimension; and a Gaussian noise layer is added after each convolution layer; at the same time, the residual block further enhances the feature expression ability of the generator, and the self-attention mechanism captures the global information of the input data; finally, the output of the generator is limited to a reasonable range through the tanh activation function.

5. The TSFS sample generation method based on an improved generative adversarial network according to claim 1, characterized in that: The linear layer contained in the discriminator described in step 2 gradually reduces the data dimension and introduces nonlinearity through the Leaky ReLU activation function; the small batch discriminant layer is used to enhance the sensitivity of the discriminator to small batch samples.

6. A TSFS sample generation method based on an improved generative adversarial network according to claim 1 or 5, characterized in that: The discriminator described in step 2 is designed to distinguish between fake samples and real samples generated by the generator; the label information is spliced ​​with the input data through the label embedding layer; in the fully connected network, nonlinear mapping is performed through multiple layers of linear transformation and LeakyReLU activation function to gradually compress the features of the input data and output a scalar; at the same time, a small batch discriminator layer is introduced to calculate the differences between samples in the batch to ensure that the discriminator can identify small differences between different generated samples.

7. The TSFS sample generation method based on an improved generative adversarial network according to claim 1, characterized in that: In the model training phase described in step 3, the generator and the discriminator are trained together through a competition mechanism. Specifically, the TSFS data described in step 1 is divided into a training set, a validation set, and a test set, and only the training data is used for data enhancement. During the training and optimization process, the generator and the discriminator use different learning rates to achieve their respective optimal effects. The generator uses a higher learning rate, and the discriminator uses a lower learning rate. During training, the update frequency of the discriminator is higher than that of the generator. rator; in each training cycle, the loss of the generator improves the generated samples by maximizing the misjudgment probability of the discriminator, while the discriminator improves its discrimination ability by maximizing the score of the real samples and minimizing the score of the generated samples; during the training process of the model, the losses of the generator and the discriminator are recorded, and the game of generation and discrimination is understood by drawing the loss curve; and after the loss curve stabilizes, the generator saves the model once every certain training epoch, and adjusts the model hyperparameters through the actual output.

8. The TSFS sample generation method based on an improved generative adversarial network according to claim 1, characterized in that: The Mahalanobis distance described in step 4 is used for the first screening of samples. The sample selection strategy in the CA-WGAN-GP optimization model first divides the original data set into a training set, a validation set, and a test set; the image data in the training set is feature extracted using a pre-trained CNN model and converted into a feature vector; the sample features are projected using linear discriminant analysis (LDA); the samples of each category are preliminarily screened according to the Mahalanobis distance; by setting a threshold, the samples far away from the center of each category in the feature space of the training set are filtered out; the deep features of each image are extracted using a pre-trained CNN model, and then the mean vector and covariance matrix of the features are calculated according to the category, and the distance between each sample and its category center is measured by the Mahalanobis distance; in order to evaluate the distribution of samples, the model calculates the Mahalanobis distance of all samples in each category, and displays the sample distribution of each category through a visualization tool, intuitively showing the samples with a large distance from the category center, which is convenient for further sample screening and optimization of the model.

9. The TSFS sample generation method based on an improved generative adversarial network according to claim 8, characterized in that: The specific operation method of step 5 is: Step 5.1: For the samples after preliminary screening, the K-means algorithm uses the same pre-trained CNN model as step 4 to extract features from the image data of the validation set; Step 5.2: Apply the K-means algorithm to these feature vectors to divide each category into multiple clusters and determine the center of each cluster; Step 5.3: Apply the K-means model trained on the validation set to the sample set after preliminary screening, and select the sample that best represents the center of each cluster from each category; Step 5.4: The selected samples constitute the final training sample set.

10. The TSFS sample generation method based on an improved generative adversarial network according to claim 9, characterized in that: The validation set and test set remain unchanged, the validation set is used for hyperparameter adjustment during model training, and the test set is used for final model evaluation.

Images