A method for identifying the impurity rate, opening rate and oil passing rate of Chinese prickly ash based on an improved U-Net network

Abstract

The present application belongs to the technical field of deep learning image segmentation, and particularly relates to a method for identifying the impurity rate, opening rate and oiling rate of Sichuan pepper based on an improved U-Net network. The present application enhances the edge details and small target segmentation capability through the improved U-Net network, expands the complex scene samples through StyleGAN and mask splicing, converts the pixel-level segmentation result into a quality index through the impurity rate evaluation model, and simultaneously discriminates the opening state and the oiling state through the lightweight convolution classifier. In the case of Sichuan pepper particle adhesion, small impurity area, similar color and texture, and low over-oiled pepper mixing ratio, the present application can improve the target region segmentation accuracy and the quality index statistical stability, reduce the subjective errors caused by manual screening and weighing, and has a broad market application prospect.

Description

Technical Field

[0001] This invention belongs to the field of deep learning image segmentation technology, and specifically relates to a method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network. Background Technology

[0002] As an important condiment, the quality indicators of Sichuan peppercorns have a significant impact on product quality and market competitiveness. Among them, the content of endogenous impurities, the opening rate, and the oil extraction rate are important indicators for measuring the quality of Sichuan peppercorns.

[0003] Impurities in Sichuan pepper mainly come from the endogenous impurities in the branches, leaves, and seeds of Sichuan pepper introduced during the harvesting, drying, and processing. According to the forestry industry standard LY / T1652-200 for Sichuan pepper quality grades, the content of endogenous impurities in Sichuan pepper products shall not exceed 17.0%, and the content of endogenous impurities in premium grade Sichuan pepper shall not exceed 4.5%.

[0004] The opening rate is also an important indicator of Sichuan pepper quality. Generally, Sichuan pepper with a higher opening rate has a more pronounced aroma and numbing sensation, while unopened Sichuan pepper often contains seeds that are not easily identified in routine visual inspections. Sichuan pepper seeds can affect the sensory quality of the pepper, increasing bitterness and off-flavors, so it is necessary to accurately determine the opening status of the Sichuan pepper.

[0005] Furthermore, there are instances in the market where fried or extracted peppercorn waste, known as "over-oiled peppercorns," is mixed with normal peppercorns for sale. Over-oiled peppercorns typically exhibit a significant decrease in aroma, color, and numbing flavor, but in small quantities, accurate identification through manual observation alone is difficult. Currently, the detection of endogenous impurities, opening rate, and over-oiling rate in peppercorns largely relies on manual screening, weighing statistics, or experience-based judgment. These methods suffer from low efficiency, high subjectivity, and poor repeatability, failing to meet the demands of modern peppercorn processing, intelligent sorting, and rapid quality testing. When the sample quantity is large, the impurity types are complex, or the proportion of over-oiled peppercorns is low, traditional manual detection methods struggle to achieve efficient, stable, and accurate identification.

[0006] With the development of machine vision and deep learning technologies, image recognition-based methods for agricultural product quality inspection are gradually being applied. U-Net is a commonly used image segmentation network that employs an encoder-decoder structure and fuses shallow detail features and deep semantic features of the image through skip connections, enabling pixel-level segmentation of the target region. However, Sichuan peppercorns are small, with significant morphological variations, and impurities, unopened peppercorns, and over-oiled peppercorns are quite similar to normal peppercorns in color and texture. Conventional image processing methods or ordinary U-Net networks are still prone to problems such as missed detections, false detections, and insufficient segmentation accuracy.

[0007] Therefore, there is a need for a method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network, so as to achieve rapid, objective, and accurate detection of these indicators, thereby improving the efficiency and quality of Sichuan pepper detection. Summary of the Invention

[0008] To address the aforementioned issues, this invention proposes a method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network. This method aims to enhance the robustness and accuracy of Sichuan pepper image segmentation models, construct and optimize evaluation models for the impurity rate, opening rate, and oil content of Sichuan pepper, thereby improving the accuracy of identifying Sichuan pepper quality indicators.

[0009] To achieve the above objectives, the technical solution of the present invention is as follows: A method for identifying impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network includes the following steps: Step 1: Collect pepper sample images and use an image annotation tool to annotate the target areas of the pepper sample images to obtain an annotated pepper sample image dataset; Preferably, in step 1, the collection of Sichuan pepper sample images involves using an industrial camera to capture 3,000 images of Sichuan pepper from different angles and under different lighting conditions. The target area includes different categories such as Sichuan pepper peel, Sichuan pepper seeds, and Sichuan pepper branches and leaves. Each image has a resolution of 2048×1080 pixels. The images are labeled using the image labeling tool LabelMe, with a total of four categories: background, Sichuan pepper peel, Sichuan pepper seeds, and Sichuan pepper branches and leaves.

[0010] Step 2: Construct an improved U-Net network for pepper image segmentation; The construction process of the improved U-Net network pepper image segmentation model in step 2 is as follows: An edge detection module is set after the last layer upsampling block of the U-Net network, and the feature map output by the last layer upsampling block is input into the edge detection module to obtain a binary edge map; the binary edge map is added to the feature map output by the last layer upsampling block to obtain the fused feature, and the fused feature is used as the output feature of the last layer upsampling block of the U-Net network to obtain the improved U-Net network pepper image segmentation model; Preferably, the edge detection module in step 2 consists of a convolutional layer and a sigmoid activation function.

[0011] Step 3: The pepper sample image dataset obtained in Step 1 is augmented using the StyleGAN method to obtain pseudo images of pepper pericarps, pepper seeds, and pepper branches and leaves. The specific steps are as follows: Image generation is performed using the StyleGAN framework, which includes a generator and a discriminator. The generator generates pseudo-images based on real images, while the discriminator determines whether the input image is real or pseudo. The network parameters are updated through adversarial training between the generator and the discriminator, enabling the trained generator to generate pseudo-images with corresponding color, texture, and morphological features. The StyleGAN model is trained on separate datasets of Sichuan pepper sample images (Sichuan pepper peel, seeds, and branches / leaves) to learn the color, texture, and morphological distributions of different target categories. During training, the generator and discriminator start with low-resolution images, gradually increasing the image resolution until pseudo-images that meet the target size requirements are generated. Preferably, in step 3, the StyleGAN model is trained using the Adam optimizer with the following parameters: the learning rates of the generator and discriminator are set to 0.002 and 0.001, respectively, and the Leaky ReLU function is used as the activation function; the dimension of all input and output latent vectors is 512, the batch size is set to 16, and a total of 50,000 iterations are trained. Through the above training method, pseudo-images in terms of shape, color, and texture can be generated.

[0012] Step 4: Generate enhanced image data based on the mask stitching method; that is, perform threshold segmentation on the pseudo image generated in step 3 to obtain the mask image of the corresponding target, and then randomly select the stitching position on the background image to stitch the pseudo image into the background image according to the mask boundary to form a new synthetic image; Preferably, the specific process in step 4 is as follows: First, the pseudo image is binarized to obtain a target region mask; then, the target mask is multiplied pixel by pixel with the corresponding region of the background image, so that the target region in the pseudo image is retained and the non-target region is removed, resulting in a processed pseudo target image; then, the processed pseudo target image is superimposed on the background image to form an enhanced image containing pepper peel, pepper seeds or pepper branches and leaves, completing one random stitching; Repeat the above random stitching operation until the preset target number is reached in the background image or there is no more space to add targets, then the operation is complete and a new composite image is obtained.

[0013] Step 5: Add the synthesized image obtained in Step 4 to the labeled Sichuan pepper sample image dataset obtained in Step 1 to obtain the expanded Sichuan pepper image semantic segmentation dataset; divide the expanded Sichuan pepper image semantic segmentation dataset into training set and test set, and input the training set into the improved U-Net network Sichuan pepper image segmentation model constructed in Step 2 for training, evaluate the model during the training process and select the optimal model parameters to obtain the optimal Sichuan pepper image segmentation model; finally, input the test set into the optimal Sichuan pepper image segmentation model to obtain the semantic segmentation results of Sichuan pepper peel, Sichuan pepper seeds and Sichuan pepper branches and leaves; Preferably, in step 5, the improved U-Net network's pepper image segmentation model uses Dice Loss as the loss function, employs the Adam optimizer to update the model parameters, sets the learning rate to 0.001, the batch size to 32, and the number of training epochs to 500; during training, the optimal model parameters are selected based on the loss value, average intersection-over-union ratio, and average precision on the training set, with a training set to test set ratio of 7:3; The formula for calculating Dice Loss is as follows: , where C represents the number of categories; Xi represents the model's prediction result for the i-th category; Yi represents the actual label for the i-th category; This represents the sum of the intersections of all classes correctly predicted by the model; These represent the sum of elements for all categories in the model prediction and the actual label, respectively.

[0014] Step 6: Based on the semantic segmentation results of the optimal pepper image segmentation model obtained in Step 5, calculate the mask area corresponding to the pepper grain region and the mask area corresponding to the impurity region, respectively; set the mass of pepper grains and impurities to be proportional to their corresponding mask areas, and establish a pepper impurity rate evaluation model; use the pepper impurity rate obtained by manual weighing and the mask areas of the pepper grain region and impurity region in the corresponding pepper image as calibration data, and solve the proportional coefficient in the pepper impurity rate evaluation model through the gradient descent algorithm; substitute the obtained proportional coefficient and the mask areas of the pepper grain region and impurity region in the pepper image to be tested into the pepper impurity rate evaluation model to obtain the impurity rate of the pepper sample to be tested.

[0015] The specific process in step 6 is as follows: In order to detect the impurity content in the peppercorns, the mass of the impurities and peppercorn particles is set to be proportional to the mask area segmented on the image, with proportionality coefficients K1 and K2 respectively, calculated using the following formula: In the formula, K1 and K2 are proportionality coefficients, and A i B represents the mask area corresponding to each peppercorn grain. iLet N be the mask area corresponding to each impurity, and M be the number of peppercorns and M be the number of impurities. To confirm the values ​​of the two proportional coefficients in the above formula, calculate the mask area corresponding to the peppercorn region and the mask area corresponding to the impurity region in each peppercorn image. At the same time, calculate the impurity rate of peppercorns in each image using a manual weighing method. Use the gradient descent algorithm to solve for the values ​​of K1 and K2, with initial values ​​of K1=1 and K2=1. The learning rate is 0.01, and the number of iterations is 1000.

[0016] Step 7: Extract the peppercorn images from the semantic segmentation results output by the optimal peppercorn image segmentation model in Step 5, scale them to a preset size, and input them into a convolutional neural network classifier to output the probability of closed peppercorns and the probability of overcooked peppercorns, thus obtaining the peppercorn opening rate and overcooking rate.

[0017] Preferably, when the obtained peppercorn images are used as input to the convolutional neural network classifier, they are scaled to a preset size of 224×224 pixels; wherein the dataset constructed from the peppercorn images is divided into a training set and a test set in a 7:3 ratio, and the convolutional neural network classifier is trained using the Adam optimizer with an initial learning rate of 0.003.

[0018] Preferably, in step 7, the convolutional neural network classifier is a three-layer convolutional neural network classifier, including three convolutional layers, a batch normalization layer, two linear layers, and a sigmoid activation function; the input image of peppercorn particles passes through the three convolutional layers sequentially, with the number of channels in each convolutional layer gradually expanding to 64, and batch normalization is performed in each convolutional layer; after processing by the two linear layers, a two-dimensional vector with a value range of 0 to 1 is output through the sigmoid activation function, and the two dimensions of the two-dimensional vector represent the probability that the peppercorn particles are closed peppercorns and fried peppercorns, respectively; the three-layer convolutional neural network classifier uses the ReLU function as the activation function and the multi-label binary classification cross-entropy loss function as the loss function.

[0019] The formula for calculating the cross-entropy loss (MultiBCELoss) in multi-label binary classification is as follows: In the formula, N is the sample size, and y ij It is the true value of the j-th label of the i-th sample, and its value ranges from 0 to 1. Let be the model's prediction of the j-th label for the i-th sample. This ensures that the model can predict the probability of multiple labels simultaneously, and the prediction of each label will affect the calculation of the loss function independently.

[0020] The beneficial effects of this invention compared to the prior art are: This invention enhances the segmentation capabilities of edge details and small targets by improving the U-Net network, expands complex scene samples through StyleGAN and mask stitching, transforms pixel-level segmentation results into quality indicators through an impurity rate evaluation model, and simultaneously distinguishes between open and over-oiled states using a lightweight convolution classifier. Under conditions of peppercorn particles sticking together, small impurity areas, similar colors and textures, and low proportion of over-oiled peppers, this invention can improve the segmentation accuracy of target regions and the statistical stability of quality indicators, while reducing subjective errors caused by manual screening and weighing. Attached Figure Description

[0021] Figure 1 Example image of collected Sichuan peppercorns; Figure 2 : Schematic diagram of the improved U-Net network structure; Figure 3 Example images of pseudo-images generated by StyleGAN, where (a) is the pericarp of a Sichuan pepper, (b) is the branches and leaves of a Sichuan pepper, and (c) is the seeds of a Sichuan pepper. Figure 4 Example image of the pseudo-image effect after background stitching; Figure 5 : Surface plot of the impurity rate evaluation model function; Figure 6 Schematic diagram of the test results for impurity content in Sichuan peppercorns; Figure 7 Schematic diagram of the neural network structure for discriminating features of Sichuan peppercorn particles; Figure 8 Schematic diagram of the loss function (a) and accuracy (b) curves of the pepper granule feature discrimination model; Figure 9 : A schematic diagram of the ROC curve of the peppercorn particle feature discrimination model, where (a) is open peppercorn and (b) is peppercorn that has been fried in oil. Detailed Implementation

[0022] The following is an illustration of an embodiment of the present invention. Figures 1 to 9 The technical solutions in the embodiments of the present invention will be described in detail below. It should be understood that the following embodiments are only used to illustrate the present invention and are not intended to limit the scope of protection of the present invention.

[0023] This embodiment provides a method for identifying the impurity rate, opening rate, and oil content of Sichuan peppercorns based on an improved U-Net network. The method first acquires and labels Sichuan peppercorn sample images, constructing a Sichuan peppercorn image segmentation model using an improved U-Net network. Then, a StyleGAN model is used to generate pseudo-images of Sichuan peppercorn pericarp, seeds, and branches / leaves, and an enhanced image is generated using a mask-based image stitching method. The enhanced image is then used to train the improved U-Net network to obtain segmentation results for the Sichuan peppercorn grain region and impurity region. Finally, a Sichuan peppercorn impurity rate evaluation model is established based on the segmentation results, and a three-layer convolutional neural network classifier is used to distinguish the opening state and oil content of the Sichuan peppercorn grains.

[0024] Example 1: Step 1: Acquire images of Sichuan peppercorns and construct a semantic segmentation dataset. Use an industrial camera to acquire images of Sichuan peppercorn samples, such as... Figure 1 As shown, the collected objects include the target areas of Sichuan pepper fruit peel, seeds, and branches and leaves. Images of Sichuan pepper were acquired from different angles and under different lighting conditions to improve the diversity of sample images and enhance the adaptability of the subsequent model to different imaging environments. In this embodiment, 3000 Sichuan pepper images were collected, each with a resolution of 2048×1080 pixels. After acquisition, the LabelMe image annotation tool was used to perform pixel-level annotation on the images, with annotation categories including background, Sichuan pepper fruit peel, seeds, and branches and leaves. After annotation, the original images and their corresponding labels were converted into the data format required for semantic segmentation model training, resulting in the annotated Sichuan pepper sample image dataset. It should be noted that LabelMe is a commonly used image annotation tool suitable for object detection and semantic segmentation tasks. In this embodiment, pixel-level annotation of the Sichuan pepper fruit peel, seeds, and branches and leaves provides accurate supervision information for subsequent improvements to the U-Net model training.

[0025] Step 2: Construct an improved U-Net network for pepper image segmentation. The U-Net network is used as the basic semantic segmentation model, consisting of an encoder and a decoder. The encoder extracts deep semantic features from the image, while the decoder restores the image's spatial resolution and outputs the segmentation result. For example... Figure 2 As shown, this embodiment adds an edge detection module after the last upsampling module of the U-Net model. The edge detection module consists of a convolutional layer and a sigmoid activation function. The feature map output from the last upsampling block is input into the edge detection module to obtain a binary edge map. The binary edge map is added to the feature map output from the last upsampling block to obtain a fused feature. The fused feature is used as the output feature of the last upsampling block of the U-Net network to obtain the improved U-Net network's pepper image segmentation model. Step 3: Construct a generative data augmentation model based on GAN. In order to improve the model's adaptability to different pepper morphologies, different impurity morphologies, and complex background conditions, this embodiment uses the StyleGAN network to perform generative augmentation on the pepper target image. The StyleGAN network includes a generator and a discriminator. The generator generates pseudo images based on real images, and the discriminator is used to determine whether the input image is a real image or a generated image.

[0026] In this embodiment, the StyleGAN model is trained separately for the peel, seeds, and branches / leaves of Sichuan pepper, enabling it to learn the color, texture, and morphological distribution of different target categories, such as... Figure 3 As shown, each sub-image displays pseudo-images of Sichuan pepper pericarp, branches and leaves, and seeds generated by the trained StyleGAN model. These pseudo-images are used for subsequent image enhancement synthesis. During training, the generator and discriminator start with low-resolution images, gradually increasing the image resolution until pseudo-images meeting the target size requirements are generated. The StyleGAN model is trained using the Adam optimizer, with a generator learning rate of 0.002 and a discriminator learning rate of 0.001. The activation function is Leaky ReLU, the latent vector dimension is 512, and the batch size is 16. A total of 50,000 iterations are performed. This training method generates pseudo-images that closely resemble real Sichuan pepper targets in shape, color, and texture.

[0027] Step 4: Generate enhanced image based on mask stitching method. First, threshold segmentation is performed on the pseudo images of pepper peel, pepper seeds, and pepper branches and leaves generated in Step 3 to obtain binary masks for the corresponding target regions. Then, the target regions in the pseudo images are retained and non-target regions are removed according to the binary masks to obtain pseudo target images.

[0028] A black background image, consistent with the image acquisition scenario of the Sichuan pepper semantic segmentation dataset, is used. A splicing position is randomly selected within the black background image, and a pseudo-target image is superimposed onto this position. This process of selecting the pseudo-target image, selecting the splicing position, and superimposing is repeated until there are no more positions in the black background image where pseudo-target images can be added, resulting in a synthesized enhanced image. Figure 4 As shown, images of Sichuan pepper fruit peel, seeds, and branches and leaves as false targets are randomly stitched onto a black background to form a synthetic enhanced image containing targets of different categories, locations, and arrangements.

[0029] In this embodiment, 10,000 enhanced images are generated using the mask stitching method described above, which are used to expand and improve the training data of the U-Net network.

[0030] Step 5: Train the improved U-Net model and obtain the optimal image segmentation model. Add the synthetic enhanced images obtained in Step 4 to the labeled Sichuan pepper sample image dataset obtained in Step 1 to obtain the expanded Sichuan pepper image semantic segmentation dataset. Divide the expanded Sichuan pepper image semantic segmentation dataset into training and testing sets with a ratio of 7:3. Input the training set into the improved U-Net network Sichuan pepper image segmentation model constructed in Step 2 for training. Evaluate the model during training and select the optimal model parameters to obtain the optimal Sichuan pepper image segmentation model. Input the testing set into the optimal Sichuan pepper image segmentation model to obtain the semantic segmentation results of Sichuan pepper peel, seeds, and branches / leaves. During training, the Adam optimizer is used to update the model parameters, with a learning rate of 0.001, a batch size of 32, and an epoch of 500. During training, the optimal model parameters are selected based on the loss value, mean intersection-over-union ratio, and mean precision on the training set. After training, input the test set images into the optimal model to obtain the semantic segmentation results of Sichuan pepper peel, seeds, and branches / leaves. Among them, the average intersection-union ratio is used to evaluate the degree of overlap between the predicted region and the real labeled region, and the average precision is used to evaluate the model's accuracy in recognizing different types of targets. Through the above evaluation methods, the segmentation effect of the improved U-Net model on different target regions in the pepper image can be comprehensively judged.

[0031] In this embodiment, Dice Loss is used as the loss function for model training. Since the background area occupies a large proportion of the pepper image, while the target areas of pepper seeds and leaves are small, there is an imbalance between categories. Therefore, using Dice Loss can improve the model's attention to small target areas, enhancing the stability and accuracy of the segmentation results. The formula for calculating Dice Loss is: , where C represents the number of categories; Xi represents the model's prediction result for the i-th category; Yi represents the actual label for the i-th category; This represents the sum of the intersections of all classes correctly predicted by the model; These represent the sum of elements for all categories in the model prediction and the actual label, respectively.

[0032] Step 6: Establish a pepper impurity rate evaluation model. The pepper image segmentation model trained in Step 5 is used to process the pepper image, obtaining masks corresponding to the pepper grain region and impurity region. The mask areas corresponding to the pepper grain region and impurity region are then calculated respectively. It is assumed that the mass of the pepper grains and impurities is proportional to the mask area obtained from segmentation in the image. Therefore, a pepper impurity rate evaluation model can be established using the mask area. Specifically, assuming that the mass of the pepper grains is proportional to their mask area and the mass of the impurities is also proportional to their mask area, the pepper impurity rate can be expressed as: In the formula, K1 and K2 are proportionality coefficients, and A i B represents the mask area corresponding to each peppercorn grain. i Let N be the number of peppercorns and M be the number of impurities, and N be the mask area corresponding to each impurity. To confirm the values ​​of the two proportional coefficients in the above formula, the mask areas corresponding to the peppercorn regions and the impurity regions in 100 peppercorn images are calculated. Simultaneously, the impurity rate of the peppercorns in each image is calculated using a manual weighing method. The gradient descent algorithm is used to solve for the values ​​of K1 and K2, with initial values ​​of K1=1 and K2=1. The learning rate is 0.01, and the number of iterations is 1000. After parameter optimization, the segmentation mask output by the improved U-Net model can be directly converted into the impurity rate detection results of the peppercorn samples. Figure 5 As shown, the objective function of the impurity rate assessment model forms a corresponding function surface as the two proportionality coefficients change. The optimization path in the figure represents the iterative process of solving for the proportionality coefficients using the gradient descent algorithm. The obtained proportionality coefficients are substituted into the pepper impurity rate assessment model to predict the impurity rate of the pepper sample, and the prediction results are compared with the results of manual weighing. Figure 6 As shown, by comparing the impurity rate predicted by the model with the impurity rate measured manually, the feasibility of impurity rate assessment based on mask area can be verified.

[0033] Step 7: Construct a peppercorn particle feature discrimination model. The peppercorn image is processed using the peppercorn image segmentation model trained in Step 5. Peppercorn particle images are extracted from the processing results and scaled to 224×224 pixels, serving as input to a three-layer convolutional neural network classifier.

[0034] The three-layer convolutional neural network classifier includes three convolutional layers, a batch normalization layer, two linear layers, and a sigmoid activation function. The image of peppercorn granules is sequentially passed through the three convolutional layers, with the number of channels gradually increasing to 64, and batch normalization is performed in each convolutional layer. After processing by the two linear layers, the sigmoid activation function outputs a two-dimensional vector with values ​​ranging from 0 to 1. The two dimensions of this two-dimensional vector represent the probabilities of the peppercorn granules being closed-mouth peppercorns and overcooked peppercorns, respectively. The three-layer convolutional neural network classifier uses the ReLU function as the activation function and a multi-label binary classification cross-entropy loss function as the loss function, calculated as follows: In the formula, N is the sample size, and y ij It is the true value of the j-th label of the i-th sample, and its value ranges from 0 to 1. Let be the model's prediction of the j-th label for the i-th sample. This ensures that the model can predict the probability of multiple labels simultaneously, and the prediction of each label will affect the calculation of the loss function independently.

[0035] like Figure 7 As shown, the peppercorn granule image is processed sequentially through three convolutional layers and batch normalization, followed by two linear layers and a sigmoid activation function, outputting the probabilities of closed-mouth peppers and overcooked peppers. A dataset is constructed using peppercorn granule samples extracted after processing by the peppercorn image semantic segmentation model, and divided into training and test sets in a 7:3 ratio. Training is performed using the Adam optimizer with an initial learning rate of 0.003. Figure 8 As shown, Figure 8 The loss function curve and accuracy curve in the model reflect the training process of the pepper particle feature discrimination model. As the number of training rounds increases, the loss value and accuracy gradually stabilize.

[0036] The image of the peppercorns to be tested is input into the trained classifier, which outputs the probability that the peppercorns are closed peppercorns or over-oiled peppercorns. Based on the output, the classifier distinguishes between open and over-oiled peppercorns. The ROC curve is used to evaluate the model's discrimination performance, and the discrimination threshold for over-oiled peppercorns is set to 0.3. In this embodiment, the classifier achieves an accuracy of 96.22% and 92.67% in distinguishing between open and over-oiled peppercorns on the test set.

[0037] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network, characterized in that, Includes the following steps: Step 1: Collect pepper sample images and use an image annotation tool to annotate the target areas of the pepper sample images to obtain an annotated pepper sample image dataset; Step 2: Construct an improved U-Net network for pepper image segmentation; The construction process of the improved U-Net network for pepper image segmentation is as follows: An edge detection module is set after the last upsampling block of the U-Net network, and the feature map output by the last upsampling block is input into the edge detection module to obtain a binary edge map; the binary edge map is added to the feature map output by the last upsampling block to obtain the fused feature, and the fused feature is used as the output feature of the last upsampling block of the U-Net network to obtain the improved U-Net network for pepper image segmentation. Step 3: The pepper sample image dataset obtained in Step 1 is augmented using the StyleGAN method to obtain pseudo images of pepper pericarps, pepper seeds, and pepper branches and leaves. The specific steps are as follows: The StyleGAN framework, which is based on style, is used to generate images. StyleGAN includes a generator and a discriminator. The generator generates pseudo images based on real images, and the discriminator is used to determine whether the input image is a real image or a pseudo image. The network parameters are updated through adversarial training between the generator and the discriminator, enabling the trained generator to generate pseudo-images with corresponding category color, texture, and morphological features. The StyleGAN model is trained separately on the pepper sample image dataset to learn the color, texture, and morphological distribution of different categories of targets. During training, the generator and discriminator start with low-resolution images and increase the image resolution as training progresses until pseudo-images that meet the target size requirements are generated. Step 4: Generate enhanced image data based on the mask stitching method; that is, perform threshold segmentation on the pseudo image generated in step 3 to obtain the mask image of the corresponding target, and then randomly select the stitching position on the background image to stitch the pseudo image into the background image according to the mask boundary to form a new synthetic image; Step 5: Add the synthesized image obtained in Step 4 to the labeled Sichuan pepper sample image dataset obtained in Step 1 to obtain the expanded Sichuan pepper image semantic segmentation dataset; divide the expanded Sichuan pepper image semantic segmentation dataset into training set and test set, and input the training set into the improved U-Net network Sichuan pepper image segmentation model constructed in Step 2 for training, evaluate the model during the training process and select the optimal model parameters to obtain the optimal Sichuan pepper image segmentation model; finally, input the test set into the optimal Sichuan pepper image segmentation model to obtain the semantic segmentation results of Sichuan pepper peel, Sichuan pepper seeds and Sichuan pepper branches and leaves; Step 6: Based on the semantic segmentation results of the optimal pepper image segmentation model obtained in Step 5, calculate the mask area corresponding to the pepper grain region and the mask area corresponding to the impurity region respectively; set the mass of pepper grains and impurities to be proportional to their corresponding mask areas, and establish a pepper impurity rate evaluation model; use the pepper impurity rate obtained by manual weighing and the mask areas of the pepper grain region and impurity region in the corresponding pepper image as calibration data, and solve the proportional coefficient in the pepper impurity rate evaluation model through the gradient descent algorithm; substitute the obtained proportional coefficient and the mask areas of the pepper grain region and impurity region in the pepper image to be tested into the pepper impurity rate evaluation model to obtain the impurity rate of the pepper sample to be tested; Step 7: Extract the peppercorn images from the semantic segmentation results output by the optimal peppercorn image segmentation model in Step 5, scale them to a preset size, and input them into a convolutional neural network classifier to output the probability of closed peppercorns and the probability of overcooked peppercorns, thus obtaining the peppercorn opening rate and overcooking rate.

2. The method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network according to claim 1, characterized in that, In step 1, the collection of Sichuan pepper sample images involves using an industrial camera to capture 3,000 images of Sichuan pepper from different angles and under different lighting conditions. The target area includes different categories such as Sichuan pepper peel, Sichuan pepper seeds, and Sichuan pepper branches and leaves. Each image has a resolution of 2048×1080 pixels. The images are labeled using the image labeling tool LabelMe, with a total of four categories labeled: background, Sichuan pepper peel, Sichuan pepper seeds, and Sichuan pepper branches and leaves.

3. The method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network according to claim 1, characterized in that, The edge detection module in step 2 consists of a convolutional layer and a sigmoid activation function.

4. The method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network according to claim 1, characterized in that, In step 3, the StyleGAN model is trained using the Adam optimizer with the following parameters: the learning rates for the generator and discriminator are set to 0.002 and 0.001, respectively, and the Leaky ReLU function is used as the activation function; the dimension of all input and output latent vectors is 512, the batch size is set to 16, and a total of 50,000 iterations are trained. Through the above training method, pseudo-images with different shapes, colors, and textures can be generated.

5. The method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network according to claim 1, characterized in that, The specific process in step 4 is as follows: First, the pseudo image is binarized to obtain a target region mask; then, the target mask is multiplied pixel by pixel with the corresponding region of the background image, so that the target region in the pseudo image is retained and the non-target region is removed, resulting in a processed pseudo target image; then, the processed pseudo target image is superimposed on the background image to form an enhanced image containing pepper peel, pepper seeds or pepper branches and leaves, completing one random stitching; Repeat the above random stitching operation until the preset target number is reached in the background image or there is no more space to add targets, then the operation is complete and a new composite image is obtained.

6. The method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network according to claim 1, characterized in that, In step 5, the improved U-Net network's pepper image segmentation model uses Dice Loss as the loss function and Adam optimizer to update the model parameters. The learning rate is set to 0.001, the batch size is set to 32, and the number of training epochs is set to 500. During training, the optimal model parameters are selected based on the loss value, average intersection-over-union ratio, and average precision on the training set. The ratio of the training set to the test set is 7:

3. The formula for calculating Dice Loss is as follows: , where C represents the number of categories; Xi represents the model's prediction result for the i-th category; Yi represents the actual label for the i-th category; This represents the sum of the intersections of all classes correctly predicted by the model; These represent the sum of elements for all categories in the model prediction and the actual label, respectively.

7. The method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network according to claim 1, characterized in that, The specific process in step 6 is as follows: In order to detect the impurity content in the peppercorns, the mass of the impurities and peppercorn particles is set to be proportional to the mask area segmented on the image, with proportionality coefficients K1 and K2 respectively, calculated using the following formula: In the formula, K1 and K2 are proportionality coefficients, and A i B represents the mask area corresponding to each peppercorn grain. i Let N be the mask area corresponding to each impurity, and M be the number of peppercorns and M be the number of impurities. To confirm the values ​​of the two proportional coefficients in the above formula, calculate the mask area corresponding to the peppercorn region and the mask area corresponding to the impurity region in each peppercorn image. At the same time, calculate the impurity rate of peppercorns in each image using a manual weighing method. Use the gradient descent algorithm to solve for the values ​​of K1 and K2, with initial values ​​of K1=1 and K2=1. The learning rate is 0.01, and the number of iterations is 1000.

8. The method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network according to claim 1, characterized in that, In step 7, when the obtained peppercorn images are used as input to the convolutional neural network classifier, they are scaled to a preset size of 224×224 pixels. The dataset constructed from the peppercorn images is divided into a training set and a test set in a 7:3 ratio. The convolutional neural network classifier is trained using the Adam optimizer with an initial learning rate of 0.

003.

9. The method for identifying the impurity rate, opening rate, and oil content of Sichuan pepper based on an improved U-Net network according to claim 1, characterized in that, In step 7, the convolutional neural network classifier is a three-layer convolutional neural network classifier, including three convolutional layers, a batch normalization layer, two linear layers, and a sigmoid activation function; the input image of peppercorn granules passes through the three convolutional layers in sequence, the number of channels of the three convolutional layers is gradually expanded to 64, and batch normalization is performed in each convolutional layer; After processing through two linear layers, a two-dimensional vector with values ​​ranging from 0 to 1 is output through the Sigmoid activation function. The two dimensions of the two-dimensional vector represent the probabilities of the peppercorns being closed-mouth peppercorns and fried peppercorns, respectively. The three-layer convolutional neural network classifier uses the ReLU function as the activation function and the multi-label binary classification cross-entropy loss function as the loss function. The formula for calculating the cross-entropy loss (MultiBCELoss) in multi-label binary classification is as follows: In the formula, N is the sample size, and y ij It is the true value of the j-th label of the i-th sample, and its value ranges from 0 to 1. Let be the model's prediction of the j-th label for the i-th sample. This ensures that the model can predict the probability of multiple labels simultaneously, and the prediction of each label will affect the calculation of the loss function independently.

Images