A tobacco vein segmentation method and device based on an improved UNet model

Abstract

The application provides a tobacco vein segmentation method and device based on an improved UNet model, and belongs to the technical field of tobacco classification and identification. The tobacco vein segmentation method comprises the following steps: collecting a hyperspectral image of a tobacco sample and constructing a sample set of the tobacco hyperspectral image; constructing a vein segmentation model based on a UNet network, wherein the vein segmentation model at least comprises an encoder, a spatial attention module and a decoder; constructing a generative adversarial network model, inputting the sample set into the vein segmentation model and training the vein segmentation model by using the generative adversarial network model; and performing vein segmentation on a hyperspectral image of a tobacco sample to be tested by using the trained vein segmentation model. The tobacco vein segmentation method and device provided by the application can maintain stable segmentation performance under different conditions, provide an efficient technical solution for tobacco quality evaluation and processing, and significantly improve the segmentation precision and robustness.

Description

Technical Field

[0001] This invention relates to the field of tobacco leaf classification and identification technology, and in particular to a method and apparatus for tobacco leaf vein segmentation based on an improved UNet model. Background Technology

[0002] Leaf veins are an important component of leaves, containing a wealth of physiological, genetic, and quality information about the plant. Tobacco is one of my country's important economic crops, and the characteristics of tobacco leaf veins play an indispensable role in the entire process of tobacco breeding, cultivation, and processing. The content of leaf veins is a crucial indicator of the economic viability of tobacco breeding and processing, while the morphology and width of leaf veins are important factors in determining leaf location during the grading process. Current methods for detecting tobacco leaf veins primarily involve manual stem removal, which results in limited sample size, poor sample representativeness, cumbersome procedures, and low accuracy. Therefore, achieving accurate and non-destructive segmentation of tobacco leaf veins is of significant practical importance.

[0003] In recent years, computer vision has seen extensive research in image segmentation. One technique involves manually annotating leaf vein information to generate leaf vein label maps, which are then fed into a U-Net network for training. This optimizes the model parameters, predicts leaf images, and quantitatively compares the results with the manually annotated label maps to obtain performance evaluation results. The encoder stage in U-Net employs multiple convolutional and pooling layers, effectively extracting abstract features from the image and capturing key information such as the shape, texture, and location of leaf veins.

[0004] However, the above method cannot accurately segment irregularly shaped leaf vein edges, such as broken, intersecting, or fused sections.

[0005] In view of this, based on years of experience in production design in this and related fields, the inventor has designed a tobacco leaf vein segmentation method and device based on an improved UNet model through repeated experiments, in order to solve the problems existing in the prior art. Summary of the Invention

[0006] The purpose of this invention is to provide a tobacco leaf vein segmentation method based on an improved UNet model, which can effectively improve the robustness and accuracy of tobacco leaf vein segmentation.

[0007] To achieve the above objectives, this invention proposes a tobacco leaf vein segmentation method based on an improved UNet network model, the tobacco leaf vein segmentation method comprising:

[0008] Collect hyperspectral images of tobacco leaf samples and construct a sample set of tobacco leaf hyperspectral images;

[0009] A leaf vein segmentation model based on the UNet network is constructed, wherein the leaf vein segmentation model includes at least an encoder, a spatial attention module, and a decoder;

[0010] A generative adversarial network model is constructed, the sample set is input into the leaf vein segmentation model, and the generative adversarial network model is used to train the leaf vein segmentation model;

[0011] The trained leaf vein segmentation model is used to segment the leaf veins in the hyperspectral image of the tobacco leaf under test.

[0012] Compared with the prior art, the present invention has the following features and advantages:

[0013] This invention proposes a method and apparatus for tobacco leaf vein segmentation based on an improved UNet model. It integrates a spatial attention mechanism to enhance the model's ability to recognize leaf vein details and optimizes the model's adaptability under complex conditions through adversarial training using a generative adversarial network. The combination of hyperspectral imaging and principal component analysis effectively improves feature separability while ensuring rapid inference, meeting the requirements of real-time processing. Generalization tests on diverse tobacco leaf samples demonstrate that the model maintains stable segmentation performance under various conditions, providing an efficient technical solution for tobacco quality evaluation and processing, and significantly improving segmentation accuracy and robustness. Attached Figure Description

[0014] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of the invention in any way. Furthermore, the shapes and proportions of the components in the drawings are merely illustrative to aid in understanding the invention and do not specifically limit the shapes and proportions of the components. Those skilled in the art, guided by the teachings of this invention, can select various possible shapes and proportions to implement the invention according to specific circumstances.

[0015] Figure 1 This is a schematic diagram of the leaf vein segmentation method of the present invention;

[0016] Figure 2 This is a schematic diagram of the hyperspectral tobacco leaf segmentation process of the present invention;

[0017] Figure 3 This is a schematic diagram of the SA-UNet network structure of the present invention;

[0018] Figure 4 This is a schematic diagram of the overall network structure of GAN-SA-UNet of the present invention;

[0019] Figure 5 This is the PCA dimensionality reduction diagram of the present invention;

[0020] Figure 6 This is a diagram illustrating the annotation of tobacco leaf veins according to the present invention.

[0021] Figure 7 This is a diagram illustrating the annotation of tobacco leaf veins according to the present invention.

[0022] Figure 8 This is a graph showing the change in segmentation loss value according to the present invention;

[0023] Figure 9 The images show the visualization results of different models of this invention. Detailed Implementation

[0024] The details of the present invention can be more clearly understood by referring to the accompanying drawings and the description of specific embodiments. However, the specific embodiments of the present invention described herein are for illustrative purposes only and should not be construed as limiting the invention in any way. Under the teachings of this invention, those skilled in the art can conceive of any possible modifications based on the invention, and these should all be considered to fall within the scope of the invention.

[0025] like Figure 1 As shown, this invention proposes a tobacco leaf vein segmentation method based on an improved UNet model. The tobacco leaf vein segmentation method includes: acquiring hyperspectral images of tobacco leaf samples and constructing a sample set of tobacco leaf hyperspectral images; constructing a leaf vein segmentation model based on a UNet network, the leaf vein segmentation model including at least an encoder, a spatial attention module, and a decoder; constructing a generative adversarial network model, inputting the sample set into the leaf vein segmentation model, and training the leaf vein segmentation model using the generative adversarial network model; and performing leaf vein segmentation on the hyperspectral image of the tobacco leaf to be tested using the trained leaf vein segmentation model.

[0026] This invention proposes a tobacco leaf vein segmentation method based on an improved UNet model. It integrates a spatial attention mechanism to enhance the model's ability to recognize leaf vein details and optimizes the model's adaptability under complex conditions through adversarial training using a generative adversarial network. The combination of hyperspectral imaging and principal component analysis effectively improves feature separability while ensuring rapid inference, meeting the requirements of real-time processing. Generalization tests on diverse tobacco leaf samples demonstrate that the model maintains stable segmentation performance under various conditions, providing an efficient technical solution for tobacco quality evaluation and processing, and significantly improving segmentation accuracy and robustness.

[0027] In an optional embodiment of the present invention, dropout convolutional blocks are used instead of traditional convolutional blocks in the encoder. This method employs structured dropout convolutional blocks instead of traditional convolutional blocks. These dropout convolutional blocks, while maintaining the effective extraction of image features by convolutional operations, introduce a random dropout mechanism in the convolutional layers, which randomly shuts down some neurons in the network, thereby preventing the model from overfitting to the training data during training. The dropout convolutional blocks randomly discard the outputs of the convolutional layers according to a predetermined probability. This allows the network to learn different feature representations during each forward propagation, effectively improving the model's generalization ability to the training data, reducing the risk of overfitting, and enabling the model to maintain high segmentation accuracy and robustness even when facing new and unseen data. Furthermore, the introduction of structured dropout ensures the spatial consistency of the feature maps, making the feature maps output by the encoder more stable in quality, providing a solid foundation for the decoder to generate accurate segmentation maps. In addition, this improved encoder structure improves segmentation accuracy without significantly increasing the computational burden, maintaining the model's efficiency, which is of great significance for achieving real-time tobacco leaf vein segmentation.

[0028] Specifically, the U-Net network is a deep learning architecture for semantic segmentation tasks. It is a foundational model in the field of image segmentation, groundbreaking in its significance and widely applied in medical image segmentation. U-Net suffers from gradient decay and loss of structural information after convolutional and pooling layers. To address these issues, a spatial and channel attention mechanism module is introduced. This module infers an attention map along the spatial dimension and multiplies it with the input feature map, thereby achieving adaptive feature refinement. Furthermore, this network employs structured dropout convolutional blocks instead of the original convolutional blocks in U-Net to prevent overfitting. These measures collectively improve the network's performance and effectively address challenges such as gradient decay and loss of structural information.

[0029] In an optional embodiment of the present invention, the spatial attention module receives the encoder output as input. Specifically, the spatial attention module is designed to further enhance the model's ability to identify key regions in an image. The spatial attention module receives the encoder output as input, and the high-level feature map output by the encoder contains image information that has undergone multi-level abstraction processing. The spatial attention module performs spatial dimension analysis and processing on these feature maps through a series of computational steps. The introduction of the spatial attention mechanism can significantly improve the model's accuracy in recognizing details of tobacco leaf veins, especially in areas with fine veins or complex textures. By adaptively adjusting the spatial weights of the feature maps, the model can focus more on key information in the image, which not only improves the quality of the segmentation results but also enhances the model's generalization ability to different tobacco leaf samples. The application of the spatial attention module also helps reduce interference from irrelevant background information, making the segmentation results clearer and more accurate.

[0030] In an alternative example of this implementation, the final layer of the leaf vein segmentation model uses a 1×1 convolution and a sigmoid activation function to generate the output segmentation map. The key step in generating the final segmentation map involves the last layer of the model. This layer employs a 1×1 convolution operation, followed by a sigmoid activation function. 1×1 convolution is an efficient feature mapping technique that can compress and adjust the number of channels in the feature map without increasing the number of parameters or computational complexity. In this invention, 1×1 convolution is used to convert the feature map output by the encoder and spatial attention module into a feature representation of the same size as the original image, while reducing the number of channels in the feature map, preparing for the final segmentation result. The sigmoid activation function is applied to these adjusted feature maps, compressing the range of feature values ​​from arbitrary real numbers to the (0,1) interval, so that the output can be interpreted as the probability that each pixel belongs to a leaf vein. Finally, based on this probability map, the location of the leaf veins can be determined, generating a clear leaf vein segmentation map. By using 1×1 convolution and the Sigmoid activation function in the last layer of the leaf vein segmentation model: the introduction of 1×1 convolution significantly improves the computational efficiency of the model because it reduces the dimension of the feature map while avoiding a significant increase in the number of parameters; the application of the Sigmoid activation function enables the model to output the probability that each pixel belongs to a leaf vein, which provides an intuitive and continuous output for subsequent segmentation decisions.

[0031] In an optional example, in the leaf vein segmentation model, the loss function is the cross-entropy function, whose formula is:

[0032]

[0033] In the formula: mn is the size of the input image, representing the number of rows and columns of the input image, respectively; x ij Represents pixels in an image; yij For a correct segmentation result, g(x) represents the value (0 or 1) of the corresponding pixel in the actual segmentation label; ij The result is the SA-UNet segmentation, representing the probability that a pixel predicted by the model belongs to a leaf vein. In the leaf vein segmentation model of this invention, the loss function employs a cross-entropy function to measure the difference between the model's predicted segmentation map and the actual segmentation label. This loss function quantifies the overall prediction error by calculating the cross-entropy for each pixel and then averaging it. By penalizing incorrect predictions, the cross-entropy loss function encourages the model to be more accurate in distinguishing leaf veins from the background, especially at leaf vein edges and in complex texture regions. During the model training process of this invention, minimizing this loss function helps the model learn more accurate segmentation features.

[0034] In an optional embodiment of the invention, in the leaf vein segmentation model, such as Figure 2 As shown, a hyperspectral image of a tobacco leaf sample is input into an encoder, which converts the hyperspectral image into high-level semantic features and outputs a high-level feature map. The high-level feature map is then input into a spatial attention module, which adjusts the spatial information weights of the high-level feature map and outputs an adjusted feature map. The adjusted feature map is then input into a decoder, which generates a segmentation map with tobacco leaf vein markings based on the adjusted feature map. In the leaf vein segmentation model of this invention, the hyperspectral image of the tobacco leaf sample is first input into the encoder. The encoder is the initial processing stage of the model, responsible for converting the raw hyperspectral image data into more abstract and high-level semantic features. The encoder's high-level semantic feature extraction enables the model to capture more subtle and abstract features in the image. These features capture key information in the image, such as shape, texture, and color, thereby improving the segmentation accuracy. The encoder extracts and compresses features through a series of convolutional and pooling layers, finally outputting a high-level feature map. These high-level feature maps are then fed into the spatial attention module. The spatial attention module adjusts the weights of spatial information in the feature maps. Its introduction significantly enhances the model's ability to identify key regions of the image, especially in areas with detailed leaf veins and complex textures. This not only improves segmentation accuracy but also enhances the model's adaptability to tobacco leaf images under different types and conditions. In this way, the model can more effectively identify and distinguish leaf veins from other parts. Finally, the adjusted feature maps are input into the decoder, which uses these feature maps to generate the final segmentation map with clear markings of the tobacco leaf veins, achieving accurate segmentation of the veins. The segmentation map generated by the decoder with clear and accurate markings of the tobacco leaf veins provides a reliable foundation for subsequent tobacco quality evaluation and processing.

[0035] In an optional embodiment of the present invention, the generative adversarial network model includes: a generator capable of adversarial training, which determines whether the data source is real data; and a discriminator, which fits real data to confuse the discriminator, making the generated data appear real. The generative adversarial network (GAN) model proposed in this invention comprises two core components: a generator and a discriminator. The generator uses random noise as input to generate data through a deep neural network. This generated data should be visually as close to real data as possible. Specifically, the generator in this invention significantly improves its data generation ability through adversarial training, and the generated data is visually almost indistinguishable from real data. This is crucial for background generation and detail enhancement in tasks such as image segmentation. The discriminator is responsible for determining whether the data is real or fake, distinguishing between the data generated by the generator and the actual real data. Because the generator's data generation ability is significantly improved after adversarial training, the discriminator's ability to distinguish between real and fake data is also strengthened, enabling more accurate identification and differentiation between generated and real data. In this adversarial training process, the generator and discriminator engage in a game-like training process. The generator continuously learns how to generate more realistic data to confuse the discriminator, while the discriminator continuously learns how to more accurately identify real and fake data. This adversarial training leads to a gradual improvement in the quality of the data generated by the generator, until the discriminator can no longer distinguish between real and fake data. As a result, in practical applications, such as tobacco leaf vein segmentation, more realistic and accurate segmentation results can be generated.

[0036] In one alternative example of this implementation, the training process of the generative adversarial network model is as follows:

[0037]

[0038] In the formula: x is the actual data, which conforms to P data (x) distribution; z represents noise data, conforming to P z (z) distribution; D(x) is the probability that x is judged as true; G(z) is the data generated by the generator; D(G(z)) is the probability that the data generated by the generator is judged as true. In the Generative Adversarial Network (GAN) model of this invention, the training process follows a specific game adversarial mechanism. This process is achieved by minimizing the loss function of the generator G and maximizing the loss function of the discriminator D. Specifically, the loss function V(D,G) consists of two parts: the first part Calculate the expected logarithm of the probability that the discriminator classifies the real data X as true; Part Two The expected logarithm of the probability that the noisy data Z generated by the generator will be classified as false by the discriminator is calculated. During training, the generator G significantly improves the quality of the generated data by continuously generating increasingly realistic data, making it visually and statistically closer to real data. This leads the generator G to attempt to generate increasingly realistic data to confuse the discriminator D, while the discriminator D continuously learns to more accurately distinguish between real and generated data. In this way, the generator and discriminator compete with each other during training. This adversarial training mechanism promotes the model's adaptability when dealing with complex data distributions, improving not only the ability to generate background and details in tasks such as image segmentation, but also providing an effective technical means for other fields requiring high-quality data generation.

[0039] In one alternative embodiment of this implementation, the leaf vein segmentation model is used as the generator, and a multi-layer convolutional network is used as the discriminator. In the Generative Adversarial Network (GAN) model of this invention, the leaf vein segmentation model acts as the generator, whose purpose is to generate realistic leaf vein segmentation images by learning features from real leaf vein segmentation data. The leaf vein segmentation model captures key features of the image through its deep learning architecture and generates highly detailed segmentation results. The multi-layer convolutional network, acting as the discriminator, is responsible for evaluating the realism of the generated segmented images. The multi-layer convolutional network utilizes its deep feature extraction capabilities to analyze the images generated by the generator and determine their similarity to real leaf vein segmentation images.

[0040] In one optional example, the generator generates fake tobacco leaf vein images and label vein images. The discriminator captures feature information from these fake images and calculates their realism. The generator learns and simulates the distribution characteristics of real tobacco leaf vein images by using random noise as input through multi-layer convolutional operations. The generated fake tobacco leaf vein images and label vein images are then provided as input to the discriminator. The discriminator, composed of multi-layer convolutional networks, is tasked with capturing feature information from these input images and evaluating their realism. By performing in-depth feature analysis on the fake images, the discriminator determines their similarity to real tobacco leaf vein images and outputs a realism score, reflecting the probability that a fake image is judged as real data. This adversarial training mechanism strengthens the game between the generator and the discriminator, driving both to achieve higher performance levels. Ultimately, the generator can produce highly realistic tobacco leaf vein images, while the discriminator possesses excellent realism detection capabilities, improving the accuracy and robustness of tobacco leaf vein segmentation.

[0041] Furthermore, random vectors or noise are input into the generator. The generator processes the random vectors and noise to generate fake tobacco vein images, which are visually similar to the labeled vein images. The fake vein images and the labeled vein images are then simultaneously input into the discriminator. The discriminator outputs a probability value, representing the probability that the input image is a real tobacco vein image. Specifically, the generator's input is random vectors or noise, which are mathematically drawn from a certain probability distribution. Using this random data, the generator, through processing by a deep neural network, generates fake tobacco vein images that are visually similar to real tobacco vein images. The generated images aim to simulate the characteristics of real tobacco veins, including shape, texture, and structure. Subsequently, these fake vein images, along with the corresponding labeled vein images, are fed into the discriminator. The discriminator is a trained multi-layer convolutional network that evaluates the features of the input images and outputs a probability value reflecting the likelihood that the input image is a real tobacco vein image. This method improves the accuracy of leaf vein segmentation, enhances the model's ability to recognize complex images, and provides an effective training strategy for image generation tasks.

[0042] In an optional embodiment of the present invention, the loss function of the generative adversarial network model adopts the cross-entropy function and is optimized using the Adam method, and its calculation formula is as follows:

[0043] loss sum =loss SA-UNet +loss uD(z) +loss Dc(z)

[0044] Where: loss SA-UNet For segmentation loss; uD(z) For discriminator loss; Dc(z) The loss function is defined as the feature loss. In the Generative Adversarial Network (GAN) model of this invention, the loss function employs the cross-entropy function and is trained using the Adam optimization algorithm. The loss function consists of three parts: segmentation loss... SA-UNet Discriminator loss uD(z) and feature loss Dc(z) Segmentation loss SA-UNet It measures the difference between the fake tobacco leaf vein image output by the generator and the real label image. Discriminator loss uD(z) This measures the accuracy of the discriminator in identifying fake images generated by the generator. Feature loss Dc(z) This measures the discriminator's ability to distinguish features between generated and real images. The sum of these three losses constitutes the total loss of the GAN model. sumThis is used to guide the training of the network. In this way, the generator learns to generate more realistic vein images, while the discriminator learns how to more accurately distinguish between real and fake images. The use of the Adam optimization algorithm further improves the efficiency and stability of model training, allowing the network parameters to converge to the optimal solution more quickly. Ultimately, this technique makes the generated fake tobacco leaf vein images visually highly similar to real images, while the discriminator has stronger discrimination capabilities, thus achieving excellent performance in tobacco leaf vein segmentation tasks.

[0045] The specific implementation process of the present invention will now be described in detail with reference to an embodiment:

[0046] 1. Materials and Methods

[0047] 1.1 Materials

[0048] This embodiment uses tobacco leaves as the main research object, collecting tobacco leaf samples of different varieties, parts, and grades from major tobacco-producing areas across the country. These include upper tobacco leaves of Sichuan K326 (B1F, B2F, B3F, B4F, B1L, B2L, B3L, B4L); middle tobacco leaves (C2F, C3F, C4F, C2L, C3L, C4L); lower tobacco leaves (X1F, X2F, X3F, X4F, X1L, X2L, X3L, X4L); Fujian Nanping Cuibi No. 1; Yunnan Yuxi Yun 87; Yunnan Chuxiong Yun 8542 grade; Guangdong Tobacco 98; Henan Sanmenxia Mianchi Zhongyan 100 C3F; and cigar wrapper tobacco leaves. The tobacco leaf dataset contains 340 flue-cured tobacco leaves, which are randomly divided into training and testing sets at an 8:2 ratio. The training set contains 272 tobacco leaves, and the validation set contains 68 tobacco leaves. The tobacco leaf samples used are shown in Table 1.

[0049] Table 1:

[0050]

[0051] The instrument uses a hyperspectral imaging system (1000–2500 nm, with 288 effective bands), SpecView software, ENVI 5.6 software, a high-performance PyCharm 2023 computer (graphics card), and a constant temperature and humidity chamber.

[0052] 1.2 Data Acquisition Methods

[0053] Before data acquisition, tobacco leaves needed to be equilibrated for 48 hours in an environment with a relative humidity of 70% and an ambient temperature of (25±2)℃ to facilitate flattening of the leaves and reduce breakage during subsequent experiments. After flattening, the tobacco leaves were placed in a hyperspectral imaging system for data acquisition. Setting the exposure time of the hyperspectral camera to 12ms and the scanning speed to 1.5cm / s helped improve the accuracy and stability of the acquired data.

[0054] 1.3 Image Correction Methods

[0055] Before data acquisition in a hyperspectral imaging system, a black and white calibration method is needed to eliminate the uneven distribution of light intensity across different wavelengths and the noise caused by dark current in the lens. A standard white calibration plate with 99% reflectivity is used to obtain a white-calibrated hyperspectral image. The light source is then turned off and the lens cap is placed on to obtain a black-calibrated hyperspectral image with 0% reflectivity. The corrected hyperspectral image R can be obtained using the formula:

[0056]

[0057] In the formula: I represents the original hyperspectral data; B represents the hyperspectral data of the black calibration plate; and W represents the hyperspectral data of the white calibration plate.

[0058] 2. Method for constructing leaf vein segmentation model

[0059] 2.1SA-UNet Network Generator

[0060] U-Net is a deep learning architecture for semantic segmentation tasks. It's a foundational model in image segmentation, groundbreaking in its application and widely used in medical image segmentation. However, U-Net suffers from gradient decay and loss of structural information after convolutional and pooling layers. To address these issues, a spatial and channel attention mechanism is introduced. This module infers an attention map along the spatial dimension and multiplies it with the input feature map, achieving adaptive feature refinement. Furthermore, this network employs structured dropout convolutional blocks instead of the original convolutional blocks in U-Net to prevent overfitting. These measures collectively improve the network's performance and effectively address challenges such as gradient decay and loss of structural information.

[0061] SA-UNet is a neural network with a U-shaped encoder-decoder structure, such as Figure 3 As shown. Its encoder contains structured dropout convolutional blocks and maxpooling operations. Each convolutional block is followed by a DropBlock, a normalization layer, and a ReLU activation function, then downsampling is performed using max pooling to increase the number of feature channels. The decoder includes deconvolutions for upsampling, halving the number of feature channels, concatenation of corresponding feature maps from the autoencoder, and structured dropout convolutional blocks. A spatial attention module is located between the encoder and decoder, and the final layer uses a 1×1 convolution and a sigmoid activation function to generate the output segmentation map. The loss function is the cross-entropy function, with the following formula:

[0062]

[0063] In the formula: mn is the input image size; y ijFor correct segmentation results; g(x) ij () represents the SA-UNet segmentation result.

[0064] SA-UNet introduces a spatial attention mechanism, enabling the model to focus more on the importance of different regions in an image. This helps to better capture key spatial features and improve segmentation accuracy in tobacco leaf vein segmentation. Compared to the traditional U-Net, the spatial attention mechanism allows SA-UNet to pay more attention to detailed information in the image, which is crucial for segmenting the fine structures and edges in tobacco leaf veins. However, introducing the attention mechanism increases the computational complexity of the model, resulting in slower training and inference speeds. Furthermore, if hyperparameters are not properly adjusted, the introduced attention mechanism can lead to overfitting on training data, resulting in poor performance on unseen data. To address these issues, this embodiment uses a generative adversarial network to optimize the SA-UNet model.

[0065] 2.2 Generative Adversarial Networks

[0066] Generative Adversarial Networks (GANs) models consist of a generator and a discriminator, which continuously compete and optimize. The discriminator determines whether the data source is real or fake, while the generator fits real data to confuse the discriminator, making the generated data appear real. The two sides train against each other in a game-like process. The training process of a GAN is as follows:

[0067]

[0068] In the formula: x is the actual data, which conforms to P data (x) distribution; z represents noise data, conforming to P z (z) distribution; D(x) is the probability that x is judged as true; G(z) is the data generated by the generator; D(G(z)) is the probability that the data generated by the generator is judged as true.

[0069] At the start of training, the generator is untrained and produces low-quality data. Therefore, the discriminator can accurately identify the source of the data, resulting in high accuracy and a large log(1-D(G(z))) value. To improve the generator's quality, the probability of the discriminator classifying the generator's data as fake needs to be reduced, thus decreasing the log(1-D(G(z))) value. This allows the generator to obtain larger gradients for model updates. The generator network continuously optimizes by gradually increasing the similarity between the generated data and the real data, and the discriminator's gradient also changes, enhancing its ability to determine the data source. The discriminator and generator are trained simultaneously, forming an adversarial relationship where they constantly compete and optimize each other.

[0070] This embodiment utilizes SA-UNet as the generator and a multi-layer convolutional network as the discriminator. The discriminator takes the generated tobacco leaf vein image and label vein image as input, performs convolution using adaptive convolutional kernels, continuously reduces the image size, and extracts features. Fully connected layers calculate the discriminator values ​​for the generated tobacco leaf vein image and label vein image. During training, the discriminator continuously improves its ability to classify generated images as fake, while the generator, receiving feedback, continuously optimizes to improve the discriminator's ability to classify them as real. These two models engage in an adversarial training process, constantly challenging each other. After training, the accuracy of the generator's image segmentation results is significantly improved, achieving accurate image segmentation results in both testing and real-world applications.

[0071] 2.3 GAN-SA-UNet Tobacco Leaf Vein Segmentation Method

[0072] To further enhance the ability of network models to segment tobacco leaf vein images, this embodiment utilizes the adversarial concept of generative adversarial networks (GAN-SA-UNet) to optimize the SA-UNet network model, proposing a GAN-SA-UNet-based tobacco leaf vein image segmentation method. In the construction of the generator and discriminator in the GAN-SA-UNet, this embodiment uses SA-UNet as the generator and employs the discriminator to enable the two networks in the model to engage in a game, thereby imposing a higher level of constraint on the SA-UNet segmentation network. The model iterates through adversarial training between the generator and discriminator, ultimately achieving the optimal segmentation model.

[0073] like Figure 4 The diagram illustrates the architecture of the GAN-SA-UNet model. The left side of the model structure diagram shows the segmentation network, and the right side shows the adversarial network. These two networks are trained adversarially. The discriminator network captures high-level feature information from the tobacco leaf vein image and distinguishes between the predicted output image and the label image from the segmentation network. If the generator's output is classified as a true value by the discriminator, this indicates the effectiveness of the generator network. The loss value for model optimization is obtained using cross-entropy. uD(z) The discriminator network extracts features from the generated segmentation results and segmentation label images, calculates the differences based on these feature maps, and then calculates the cross-entropy loss between the images. Dc(z) The model's loss function is optimized using the Adam method.

[0074] loss sum =loss SA-UNet +loss uD(z) +loss Dc(z)

[0075] Where: loss SA-UNet For segmentation loss; uD(z)For discriminator loss; Dc(z) This is the characteristic loss.

[0076] In the adversarial learning between the generator and discriminator, the network is updated by calculating the loss function of the discriminator, features, and SA-UNet, thus optimizing the entire network. GAN-SA-UNet introduces adversarial training, making the generated segmentation results more realistic and accurately reflecting the details and textures of tobacco leaf veins. The robustness of the generative adversarial network improves the model's segmentation performance under different conditions and enhances its generalization ability. Through global adversarial training, the generator can better understand the overall image structure of tobacco leaf veins, resulting in more consistent and reasonable segmentation results. GAN-SA-UNet, which integrates generative and discriminative networks, can adapt to complex tobacco leaf veins, effectively solve the problem of vein breakage, and produce smooth and clear segmentation results, providing a reliable solution for tobacco leaf image processing.

[0077] result

[0078] The first three principal components of the spectral dimension of the hyperspectral image are combined into a 3-channel image, such as... Figure 5 As shown, compared with ordinary RGB images, the differences between the leaf surface and veins in tobacco leaf images after spectral PCA dimensionality reduction are more significant, and the vein features are more prominent. This is because PCA dimensionality reduction helps to highlight the main change directions related to the leaf surface and veins in the hyperspectral data of tobacco leaves, which will help improve the accuracy of subsequent segmentation tasks.

[0079] High-quality data annotation of tobacco leaf veins is a crucial step in accurate segmentation. The original hyperspectral data was dimensionality-reduced using PCA to synthesize a pseudo-color image. Using the pixel function of the Region of Interest (ROI) tool in ENVI 5.6 software, the synthesized pseudo-color image was imported, and the main vein and secondary vein regions of the tobacco leaves were manually annotated. The pixel-by-pixel annotation of the tobacco leaf veins was generated into an XML file, and the annotated map of the tobacco leaf veins was obtained after converting the XML file. The annotation results are shown below. Figure 6 As shown.

[0080] To improve the robustness, transferability, and more accurate segmentation of the network model, this embodiment selects typical varieties and grades of tobacco leaves and applies random flipping, Gaussian blurring, affine transformation, image cropping, and padding to the dataset, enabling the dataset to better fit the proposed network. This data augmentation helps the model better learn different features of the data, improves the model's generalization ability, and allows it to better process tobacco leaf images in various practical application scenarios.

[0081] 3.2 Model Improvement and Optimization Training

[0082] In the tobacco leaf vein segmentation task, the base model is U-Net, which has been improved and optimized. Specifically, the following two main strategies are adopted to improve model performance: adding a spatial attention mechanism and GAN optimization. To verify the performance differences between different networks and to verify the impact of adding GAN optimization on the tobacco leaf vein segmentation effect, some network training hyperparameters such as optimization method, number of training epochs, and Dropout rate are set as shown in Table 2.

[0083] Table 2:

[0084]

[0085] 3.3 Application Validation

[0086] like Figure 8 As shown, the segmentation visualization results of the network model used in this embodiment are presented. The spatial attention mechanism introduced by GAN-SA-UNet can better capture key spatial features, focusing on the fine structures and edges in tobacco leaf veins, and can accurately capture complex texture features, resulting in more accurate segmentation results with clearer edges, which are closer to the ground truth.

[0087] To further verify the robustness and generalization ability of the GAN-SA-UNet leaf vein segmentation model, the optimal model weights were trained and validated on samples outside the training and test sets. Extensive testing was conducted on tobacco leaves from different origins (Yunnan, Fujian, Sichuan, Henan), different parts (upper, middle, lower), different varieties (K326, Yun 87, Yun 85, Yueyan 98, Cuibi No. 1), and different types (flue-cured tobacco, cigar tobacco). Some segmentation results are shown above. Figure 9 As shown, the model exhibits a certain tolerance for diverse tobacco leaf images, maintaining stable segmentation performance even under uncertainties and complex environments, demonstrating strong generalization ability. However, some segmented leaf vein images show discontinuity, meaning the veins are not continuous. This is due to leaf breaks and occlusion caused by leaf wrinkles, preventing the veins from being fully exposed. This phenomenon is difficult to avoid in complex real-world applications. The next step will utilize image inpainting techniques from deep learning to reconstruct the segmented broken veins, ensuring the continuity of the entire tobacco vein network.

[0088] 4. Conclusion

[0089] To achieve accurate segmentation of fine leaf veins under complex leaf texture conditions, PCA dimensionality reduction was performed on the first three principal components of hyperspectral imaging to synthesize a 3-channel image. The PCA interpretation rate of the first three principal components of leaf vein and leaf surface spectra exceeded 95.5%, and the spectral features after dimensionality reduction showed obvious separability. The synthesized image of the first three principal components can highlight the differences between leaf surface and leaf veins, making the leaf vein features more prominent.

[0090] The GAN-SA-UNet model helps capture key spatial features in tobacco leaf vein segmentation tasks. By introducing GAN network optimization training, the network can learn a wider range of contextual information. It achieves a breakthrough compared to traditional methods, which suffer from slow convergence speed, unstable training process, blurred leaf vein edge segmentation, and missed segmentation. The IoU achieved is 66.23%, and the inference time for a single image is 4ms.

[0091] Validation tests were conducted on tobacco leaves from different origins, parts, grades, and types. Even under uncertainties and complex environments, the model maintained stable segmentation performance, reflecting the details and texture of the veins, achieving accurate segmentation, and demonstrating strong generalization ability, meeting the performance requirements for industrial applications. This method provides a new approach and solution for tobacco leaf vein segmentation, offering technical support and assurance for applications such as tobacco quality evaluation, tobacco classification, and tobacco processing.

[0092] The detailed explanations of the above embodiments are intended only to explain the present invention so as to facilitate a better understanding of the present invention. However, these descriptions should not be construed as limiting the present invention for any reason. In particular, the various features described in different embodiments can be arbitrarily combined with each other to form other embodiments. Unless there is an explicit description to the contrary, these features should be understood to be applicable to any embodiment, and not limited to the described embodiments.

Claims

1. A method for segmenting tobacco leaf veins based on an improved UNet network model, characterized in that, The method for segmenting tobacco leaf veins includes: Collect hyperspectral images of tobacco leaf samples and construct a sample set of tobacco leaf hyperspectral images; A leaf vein segmentation model based on the UNet network is constructed, wherein the leaf vein segmentation model includes at least an encoder, a spatial attention module, and a decoder; A generative adversarial network model is constructed, the sample set is input into the leaf vein segmentation model, and the generative adversarial network model is used to train the leaf vein segmentation model; The trained leaf vein segmentation model is used to segment the leaf veins of the tested tobacco leaf from its hyperspectral image. In the encoder, dropout convolutional blocks are used instead of the original convolutional blocks; The leaf vein segmentation model is used as a generator, and a multi-layer convolutional network is used as a discriminator. The generator generates fake tobacco leaf vein images and label vein images, and the discriminator captures feature information in the fake tobacco leaf vein images and label vein images, and calculates the authenticity of the fake tobacco leaf vein images and label vein images. A random vector or noise is input into the generator, which processes the random vector and noise to generate the fake tobacco leaf vein image, which is visually similar to the label vein image. The fake vein image and the labeled vein image are simultaneously input into the discriminator, and the discriminator outputs a probability value, which represents the probability that the input image is a real tobacco leaf vein image. In the generative adversarial network model, the loss function adopts the cross-entropy function and is optimized using the Adam method. Its calculation formula is as follows: In the formula: For segmentation loss; For discriminator loss; This is the characteristic loss.

2. The tobacco leaf vein segmentation method based on the improved UNet network model as described in claim 1, characterized in that, The spatial attention module receives the encoder's output as input.

3. The tobacco leaf vein segmentation method based on the improved UNet network model as described in claim 2, characterized in that, In the leaf vein segmentation model, the last layer uses a 1×1 convolution and a sigmoid activation function to generate the output segmentation map.

4. The tobacco leaf vein segmentation method based on the improved UNet network model as described in claim 3, characterized in that, In the leaf vein segmentation model, the loss function is the cross-entropy function, and its formula is: In the formula: x is the input image size; ij Represents pixels in an image; For correct segmentation results; The segmentation result is from SA-UNet.

5. The tobacco leaf vein segmentation method based on the improved UNet network model as described in claim 1, characterized in that, In the leaf vein segmentation model described above The hyperspectral image of the tobacco leaf sample is input into the encoder, which converts the hyperspectral image into high-level semantic features and outputs a high-level feature map. The high-level feature map is input into the spatial attention module, which adjusts the spatial information weights of the high-level feature map and outputs the adjusted feature map. The adjusted feature map is input into the decoder, which generates a segmentation map with tobacco leaf vein markings based on the adjusted feature map.

6. The tobacco leaf vein segmentation method based on the improved UNet network model as described in claim 1, characterized in that, The generative adversarial network model includes those capable of game-based adversarial training: The generator determines whether the data source is real or fake. The discriminator fits real data to confuse the discriminator, making the generated data appear real.

7. The tobacco leaf vein segmentation method based on the improved UNet network model as described in claim 6, characterized in that, The training process of the generative adversarial network model is as follows: In the formula: This is real data and conforms to... distributed; This is noisy data, which conforms to... distributed; for The probability of being judged as true; Generate data for the generator; The probability that the data generated by the generator is judged as true.

Images