A wild ginseng grade identification method and system based on an attention mechanism feature pyramid network

Abstract

The present application relates to a kind of wild ginseng grade identification method and system based on attention mechanism feature pyramid network, comprising: picture preprocessing: the operation such as cutting, rotation and transposition is executed to picture, and wild ginseng direction is normalized;Picture augmentation: the operation such as background replacement, flip is executed to picture, input feature is improved, and model robustness is increased;Network model is built and training: input wild ginseng picture, extract wild ginseng feature matrix;Network output post-processing: the wild ginseng feature matrix that model outputs is spliced, normalizes, and selects maximum index as label value;Model evaluation: the performance of model is evaluated using verification set;Inference deployment: the inference performance of model is improved, is deployed.This application adds artificial neural network in wild ginseng grade identification technology, greatly improves the generalization ability of grade identification, improves the robustness of model.The method provided by the present application is simple, efficient, reusable, can reduce labor cost, time cost.

Description

Technical Field

[0001] This invention belongs to the field of image processing and application technology, and relates to a method and system for identifying the grade of wild ginseng based on a feature pyramid network with an attention mechanism. Background Technology

[0002] Wild ginseng typically grows to be several decades or even hundreds of years old. It is a pristine ginseng that grows in deep mountain forests and is considered the highest quality among ginseng varieties, possessing the highest ginsenoside content and extremely limited quantity. Wild ginseng is believed to have numerous medicinal properties, including replenishing vital energy, nourishing blood, reducing inflammation and fighting cancer, calming the mind and strengthening the body, beautifying the skin and promoting longevity, darkening hair and preventing hair loss, and treating diabetes. Wild ginseng of different years, growing environments, and shapes can be classified into different grades, each with significantly different medicinal properties and economic value. Currently, ginseng varieties are primarily identified based on morphological, microscopic characteristics, and physicochemical properties.

[0003] In the industry, the key to accurate and rapid grading of wild ginseng lies in the rapid and accurate extraction of the five elements and six body characteristics of wild ginseng while simultaneously determining its grade. Currently, artificial intelligence (AI) technology is being used more and more widely, and applying AI to wild ginseng grading can significantly improve efficiency and accuracy. The generalization ability and robustness of AI in extracting the physical characteristics of wild ginseng is also crucial for the shift from manual to digital technology in wild ginseng grading. However, existing AI technologies for wild ginseng grading suffer from problems such as slow positioning speed, inaccurate positioning, and a lack of universality in the identification models. For example, current AI methods for grading wild ginseng with significant physical differences face the following challenges: 1) Wild ginseng grading is achieved by identifying the five elements and six body characteristics. The first challenge is how to quickly locate and focus on these characteristics. 2) Wild ginseng varies in quality, and the sizes of its five elements and six body characteristics differ. The second challenge is how to quickly extract these characteristics at different visual scales during grading. 3) Wild ginseng grows unevenly, and its five elements and six body characteristics are not the same. The third difficulty in the grading of wild ginseng is how to establish a universal wild ginseng grading model with limited data.

[0004] Therefore, there is an urgent need for a wild ginseng grading technology that can overcome the above-mentioned problems in existing technologies and solve the corresponding technical difficulties. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method and system for grading wild ginseng based on an attention mechanism feature pyramid network. This solution mainly adopts the following technical means:

[0006] 1) Attention mechanism artificial neural network to achieve rapid localization, focusing on the five elements and six body characteristics of wild ginseng.

[0007] The attention mechanism is to filter out a small amount of important information from a large amount of information, focusing on important features and ignoring irrelevant and redundant information.

[0008] This invention uses a channel- and space-based attention mechanism. Global average pooling in the network provides feedback to every pixel in the wild ginseng feature map, while global max pooling provides feedback to the location with the largest response in the wild ginseng feature map during gradient backpropagation calculation, thereby achieving rapid localization. This invention focuses on solving the key technical challenge of the research on the five elements and six bodies characteristics of wild ginseng.

[0009] 2) Feature pyramid artificial neural network, which can integrate the features of the Five Elements and Six Bodies at different visual scales.

[0010] The Feature Pyramid Artificial Neural Network (APNN) integrates feature maps of different scales at each layer, possessing strong semantic information. It can fuse low-resolution feature maps with strong semantic information and high-resolution feature maps with weaker semantic information but rich spatial information, with minimal computational increase. Its unique feature lies in its use of a feature pyramid structure to pass features from shallow to deep layers, fusing shallow and high-level feature maps while simultaneously predicting each layer's feature maps separately. This addresses the challenge of convolutional neural networks where semantic information increases with depth, but feature maps become smaller and lower in resolution, and deep feature maps tend to overlook small targets.

[0011] 3) By combining attention mechanism artificial neural network and feature pyramid artificial neural network, an artificial neural network for wild ginseng grading is constructed. The generalization ability and robustness of the artificial neural network can ensure the universality of this invention in wild ginseng grading.

[0012] Artificial neural networks possess many impressive properties, among which robustness and generalization ability are particularly outstanding. Robustness refers to the system maintaining stability and effectiveness even when certain parameters change slightly or the control input deviates slightly from its optimal value; generalization ability refers to the network model's good predictive ability for other variable domains based on a limited sample. Both exist simultaneously in artificial neural network models. This invention achieves excellent generalization ability and robustness in a wild ginseng grading model by manually configuring hyperparameter control, training network weights, and data augmentation.

[0013] To achieve the above objectives, the present invention provides the following technical solution:

[0014] A method for grading wild ginseng based on an attention mechanism feature pyramid network, comprising the following steps:

[0015] S1. Image preprocessing: Perform operations on the image including cropping, rotation and transposition, and normalize the orientation of the wild ginseng;

[0016] S2, Image Augmentation: Performs operations on images, including background replacement and flipping, to enhance input features and increase model robustness;

[0017] S3. Build and train the network model: Input wild ginseng images and extract the wild ginseng feature matrix;

[0018] S4. Post-processing of network output: The wild ginseng feature matrix output by the model is concatenated and normalized, the maximum index is selected as the label value, multi-scale output is used, the modulo of the number of grades is taken, and finally the true grade value is obtained.

[0019] S5. Model Evaluation: The performance of the model is evaluated using a validation set; this method uses mAp as the evaluation metric.

[0020] S6. Inference Deployment: Deploy the model for inference to improve inference performance.

[0021] Furthermore, in step S1, before the image is input into the model, the input image is preprocessed. The purpose of this is to reduce the size of the background feature region, thereby increasing the proportion of wild ginseng features in the input, allowing the model to focus more on extracting wild ginseng features, reducing unnecessary computation, and significantly improving inference speed. Specifically, this includes:

[0022] S11. Read the image from local storage;

[0023] S12. Convert to grayscale. There are two methods for this step: converting a color image to grayscale or selecting one of the channels. This method uses the channel method and selects the G channel as the grayscale image.

[0024] S13. Perform binarization. For the grayscale image in the previous step, set the pixel value greater than or equal to 90 to 255, and set the pixel value less than 90 to 0. At this point, you will get a black and white image containing only 0 and 255 pixels.

[0025] S14. Target region identification: Select the image from step S13 and extract the largest white area as the feature region of wild ginseng.

[0026] S15. Extract the coordinate points of the wild ginseng feature area from step S14, select the minimum index of the upper left corner and the maximum index of the lower right corner to obtain the diagonal coordinate points of a rectangular area;

[0027] S16. Cropping the ginseng target: Using the diagonal coordinates of step S15, crop a rectangular area from the image read in step S11. At this point, you get an image containing the ginseng after removing the background.

[0028] S17. Compare the height and width of the resulting image from step S16. If the height is less than the width, transpose the image once; otherwise, skip this step.

[0029] S18. Based on the comparison and calculation in the previous step, save an image with a height greater than its width. At this moment, we obtain image data of a wild ginseng placed vertically.

[0030] Furthermore, in step S2, during the initial stage of data collection, the amount of usable data is usually limited. To ensure the smooth progress of model training, data augmentation is used to increase the amount of usable data, specifically including the following steps:

[0031] S21. Read the cropped image;

[0032] S22. Convert to grayscale. There are two methods for this step: convert the color image to grayscale or select one of the channels. This method uses the channel method and selects the G channel as the grayscale image.

[0033] S23. Perform binarization operation. For the grayscale image in the previous step, set the pixel value greater than or equal to 200 to 1 and the pixel value less than 200 to 0. At this time, a single-channel feature matrix containing only 0 and 1 pixels will be obtained, with the height and width consistent with the input image.

[0034] S24. Traverse the feature matrix obtained in step S23, obtain the index of the pixel value equal to 1, change the pixel value of all channels of the image read in step S21, and the index of the position to be changed is the index obtained in this step.

[0035] S25. The background to be replaced includes, but is not limited to, solid colors, including replacing all with (255,0,0), (0,255,127), or generating each index point randomly.

[0036] S26. Save a new image. At this moment, although the wild ginseng remains unchanged, the diversity of background features of the input image is improved, which promotes the generalization ability of the model.

[0037] Furthermore, in step S3, a network model is built and trained. This model incorporates key technologies such as attention mechanisms, upsampling, and downsampling, specifically including:

[0038] S31. Read the image and set the height of the input image to 960 and the width to 640 to ensure that most images are less distorted after scaling.

[0039] S32. Perform normalization and channel pre-processing on the image obtained in the previous step according to the following formula:

[0040]

[0041] In the formula, Pixel chw Represents the original pixel;

[0042] S33. Perform convolution according to formula 3, batch normalization according to formula 2, and ReLU activation operation on the result of step S32 in sequence;

[0043] ReLU(x) = max(0,x) (Formula 2)

[0044] In Formula 2, max(0,x) represents the maximum value between 0 and x;

[0045]

[0046] In formula 3, n h n represents the height of the input feature matrix. w k represents the width of the input feature matrix. h k represents the height of the convolution kernel. w represents the width of the convolution kernel, p represents the width of the expansion, and s represents the stride of the convolution kernel.

[0047] S34. First, use Formula 4 (channel attention mechanism) to extract channel features from the result of step S33. Then, use Formula 5 (spatial attention mechanism) to extract spatial features from the output.

[0048] ChannelAttention(x) = x × [avg_pool(x) + max_pool(x)] (Formula 4)

[0049] In Equation 4, avg_pool(x) represents performing average pooling on x, and max_pool(x) represents performing max pooling on x.

[0050] SpatialAttention(x) = x × [mean(x), max(x)] Formula 5

[0051] In Formula 5, mean(x) represents the average value of x, and max(x) represents the maximum value of x.

[0052] S35, 2x upsampling, maps low-dimensional features to high-dimensional features, allowing the network to maintain high-dimensional features while also paying attention to small target features; bilinear interpolation is used to achieve upsampling, which is computationally more complex than other interpolation methods, but is negligible compared to transposed convolution, thus maintaining the advantage of computational speed while fusing features.

[0053] The beneficial effects of this invention are as follows: This invention provides a wild ginseng grading technology based on an attention mechanism feature pyramid network. By incorporating an artificial neural network into the wild ginseng grading technology, the generalization ability of grading is greatly improved. Switching between different solid colors and random background colors in the training set of wild ginseng data significantly enhances the robustness of the model. Compared to traditional manual grading, the method provided by this invention is simple, efficient, and reusable, reducing labor and time costs. The method has a wide range of applications, regardless of time or environment, and is available 24 hours a day. This method can quickly and accurately determine the grade of wild ginseng, facilitating its rapid entry into the market and generating economic benefits.

[0054] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0055] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0056] Figure 1 This is an overall flowchart of the method described in this invention;

[0057] Figure 2 Here is a flowchart of the image preprocessing process;

[0058] Figure 3 Flowchart for image augmentation;

[0059] Figure 4 Network structure diagram;

[0060] Figure 5 Post-processing flowchart for network output;

[0061] Figure 6 This is the curve showing the change in training loss. Detailed Implementation

[0062] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0063] Figure 1The overall flowchart of the method described in this invention is shown in the figure. The main process of the wild ginseng grading method based on the attention mechanism feature pyramid network provided by this invention is as follows: S1, Image preprocessing: Perform operations including cropping, rotating, and transposing on the image to normalize the orientation of the wild ginseng; S2, Image augmentation: Perform operations including background replacement and flipping on the image to enhance the input features and increase the robustness of the model; S3, Build and train the network model: Input wild ginseng images and extract the wild ginseng feature matrix; S4, Post-processing of network output: Concatenate and normalize the wild ginseng feature matrix output by the model, select the maximum index as the label value, use multi-scale output, take the modulo of the number of grades, and finally obtain the true grade value; S5, Model evaluation: Use the validation set to evaluate the performance of the model; this method uses accuracy as the evaluation index; S6, Inference deployment: Deploy the model for inference to improve inference performance.

[0064] Specifically, in step S1, the image preprocessing stage, before the image is input into the model, this method requires preprocessing the input image. The purpose of this is to reduce the size of the background feature region, thereby increasing the proportion of wild ginseng features in the input, allowing the model to focus more on extracting wild ginseng features, reducing unnecessary computation, and significantly improving inference speed. See below for detailed execution steps. Figure 2 The steps are explained in detail below:

[0065] (1) Read the image from local storage;

[0066] (2) Convert to grayscale. There are two methods for this step: converting a color image to grayscale or selecting one of the channels. This method uses the channel method and selects the G channel as the grayscale image;

[0067] (3) Perform binarization operation. For the grayscale image in step (2), the pixel value greater than or equal to 90 points is set to 255, and the pixel value less than 90 points is set to 0. At this time, a black and white image containing only 0 and 255 pixels will be obtained.

[0068] (4) Target region identification: Select the largest white region from step (3) as the feature region of wild ginseng;

[0069] (5) Extract the coordinates of the wild ginseng feature area in step (4), select the minimum index of the upper left corner and the maximum index of the lower right corner to obtain the diagonal coordinates of a rectangular area;

[0070] (6) Crop the ginseng target. Using the diagonal coordinates of step (5), crop a rectangular area from the image read in step (1). At this point, you will get an image containing the ginseng after removing the background.

[0071] (7) Compare the height and width of the resulting image in step (6). If the height is less than the width, transpose the image once; otherwise, skip this step.

[0072] (8) By comparing and calculating in step (7), save an image with a height greater than its width. At this moment, we obtain an image of wild ginseng placed vertically.

[0073] In the image augmentation stage of step S2, during the initial data acquisition phase, the available data is usually limited. To ensure smooth model training, data augmentation is typically used in experiments to increase the amount of usable data. See the execution steps below. Figure 3 The steps are explained in detail below:

[0074] (1) Read the cropped image;

[0075] (2) Convert to grayscale. There are two methods for this step: converting a color image to grayscale or selecting one of the channels. This method uses the channel method and selects the G channel as the grayscale image;

[0076] (3) Perform binarization operation. For the grayscale image in step (2), the pixel values ​​greater than or equal to 200 are set to 1, and the pixels less than 200 are set to 0. At this time, a single-channel feature matrix containing only 0 and 1 pixels with the same height and width as the input image will be obtained.

[0077] (4) Traverse the feature matrix obtained in step (3), obtain the index of the pixel value equal to 1, change the pixel value of all channels of the image read in step (1), and the index of the position to be changed is the index obtained in this step.

[0078] (5) The background to be replaced can be a solid color, such as replacing all with (255,0,0), (0,255,127), or generating each index point randomly.

[0079] (6) Save a new image. At this point, although the wild ginseng remains unchanged, the diversity of background features of the input image is enhanced, which promotes the generalization ability of the model.

[0080] In step S3, during the network model building and training phase, this method employs an attention-based pyramid network for wild ginseng grading. The model incorporates key technologies such as attention mechanisms, upsampling, and downsampling. Its network structure is shown below. Figure 4 The key technologies are detailed below:

[0081] (1) Read the image and set the height of the input image to 960 and the width to 640 to ensure that most images are less deformed after scaling;

[0082] (2) Perform normalization (Formula 1) and channel pre-positioning operations on the image obtained in step (1);

[0083]

[0084] In Formula 1, Pixel chw Represents the original pixel.

[0085] (3) Perform convolution (Formula 3), batch normalization and ReLU (Formula 2) activation operations on the results of step (2) in sequence;

[0086] ReLU(x) = max(0,x) (Formula 2)

[0087] In Formula 2, max(0,x) represents the maximum value between 0 and x.

[0088]

[0089] In formula 3, n h n represents the height of the input feature matrix. w k represents the width of the input feature matrix. h k represents the height of the convolution kernel. w represents the width of the convolution kernel, p represents the width of the expansion, and s represents the stride of the convolution kernel.

[0090] (4) First, use the channel attention mechanism (Formula 4) to extract channel features from the results of step (3), and then use the spatial attention mechanism (Formula 5) to extract spatial features from the output;

[0091] ChannelAttention(x) = x × [avg_pool(x) + max_pool(x)] (Formula 4)

[0092] In Equation 4, avg_pool(x) represents performing average pooling on x, and max_pool(x) represents performing max pooling on x.

[0093] SpatialAttention(x) = x × [mean(x), max(x)] (Formula 5)

[0094] In Formula 5, mean(x) represents the average value of x, and max(x) represents the maximum value of x.

[0095] (5) Double upsampling maps low-dimensional features to high-dimensional features, allowing the network to maintain high-dimensional features while also paying attention to small target features. This invention uses bilinear interpolation to achieve upsampling. Although computationally more complex than other interpolation methods, it is negligible compared to transposed convolution computation, maintaining the advantage of computational speed while fusing features.

[0096] Figure 5The post-processing flowchart for network output is shown in the figure. The model output will undergo further processing, and the processing steps are as follows:

[0097] (1) Expand the output matrix so that the dimension is 1;

[0098] (2) Perform full connection, with the number of neurons equal to the number of classifications required, as a hyperparameter setting;

[0099] (3) Concatenate the results from the previous step to obtain a vector with dimensions (1, 4 × number of categories);

[0100] (4) Obtain the maximum value of the vector with dimension (1, 4 × number of categories), which is the highest score;

[0101] (5) Obtain the index with the highest score, divide the index by the number of categories and take the remainder, the remainder is the category level label.

[0102] Figure 6 This is the training loss curve. This model uses cross-entropy to calculate the training loss function between the model output and the labels, as shown in the following formula:

[0103]

[0104] In the formula, k represents the number of categories, and y represents the actual labels. This indicates the model's output label.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications should be covered within the scope of the claims of the present invention.

Claims

1. A method for grading wild ginseng based on a feature pyramid network with an attention mechanism, characterized in that: The method includes the following steps: S1. Image preprocessing: Perform operations on the image including cropping, rotation and transposition, and normalize the orientation of the wild ginseng; S2, Image Augmentation: Performs operations on images, including background replacement and flipping, to enhance input features and increase model robustness; S3. Build and train the network model: Input wild ginseng images and extract the wild ginseng feature matrix; S4. Post-processing of network output: The wild ginseng feature matrix output by the model is concatenated and normalized, the maximum index is selected as the label value, multi-scale output is used, the modulo of the number of grades is taken, and finally the true grade value is obtained. S5. Model Evaluation: Evaluate the model's performance using a validation set; S6. Inference Deployment: Deploy the model for inference to improve inference performance; In step S1, before the image is input into the model, the input image is preprocessed, specifically including: S11. Read the image from local storage; S12. Convert to grayscale using the channel method, and select channel G as the grayscale image; S13. Perform binarization. For the grayscale image in the previous step, set the pixel value greater than or equal to 90 to 255, and set the pixel value less than 90 to 0. At this point, you will get a black and white image containing only 0 and 255 pixels. S14. Target region identification: Select the image from step S13 and extract the largest white area as the feature region of wild ginseng. S15. Extract the coordinate points of the wild ginseng feature area from step S14, select the minimum index of the upper left corner and the maximum index of the lower right corner to obtain the diagonal coordinate points of a rectangular area; S16. Cropping the ginseng target: Using the diagonal coordinates of step S15, crop a rectangular area from the image read in step S11. At this point, you get an image containing the ginseng after removing the background. S17. Compare the height and width of the resulting image from step S16. If the height is less than the width, transpose the image once; otherwise, skip this step. S18. Based on the comparison and calculation in the previous step, save an image with a height greater than its width. At this moment, we obtain image data of a wild ginseng placed vertically. In step S2, the amount of usable data is increased using data augmentation, specifically including the following steps: S21. Read the cropped image; S22. Convert to grayscale using the channel method, and select channel G as the grayscale image; S23. Perform binarization operation. For the grayscale image in the previous step, set the pixel value greater than or equal to 200 to 1 and the pixel value less than 200 to 0. At this time, a single-channel feature matrix containing only 0 and 1 pixels will be obtained, with the height and width consistent with the input image. S24. Traverse the feature matrix obtained in step S23, obtain the index of the pixel value equal to 1, change the pixel value of all channels of the image read in step S21, and the index of the position to be changed is the index obtained in this step. S25. The background to be replaced includes solid colors, all of which are replaced with (255,0,0), (0,255,127), or each index point is randomly generated individually. S26. Save a new image. At this moment, although the wild ginseng remains unchanged, the diversity of background features of the input image is improved, which promotes the generalization ability of the model. In step S3, the network model is built and trained, specifically including: S31. Read the image and set the height of the input image to 960 and the width to 640 to ensure that the ginseng is not deformed after the image is scaled. S32. Perform normalization and channel pre-processing on the image obtained in the previous step according to the following formula: In the formula, Pixel chw Represents the original pixel; S33. Perform convolution according to formula 3 and ReLU activation according to formula 2 on the result of step S32 in sequence; ReLU(x) = max(0,x) (Formula 2) In Formula 2, max(0,x) represents the maximum value between 0 and x; In formula 3, n h n represents the height of the input feature matrix. w k represents the width of the input feature matrix. h k represents the height of the convolution kernel. w represents the width of the convolution kernel, p represents the width of the expansion, and s represents the stride of the convolution kernel. S34. First, use Formula 4 (channel attention mechanism) to extract channel features from the result of step S33. Then, use Formula 5 (spatial attention mechanism) to extract spatial features from the output. ChannelAttention(x)=x×[avg_pool(x)+max_pool(x)] In Formula 4, avg_pool(x) represents performing average pooling on x, and max_pool(x) represents performing max pooling on x. SpatialAttention(x) = x × [mean(x), max(x)] (Formula 5) In Formula 5, mean(x) represents the average value of x, and max(x) represents the maximum value of x. S35, 2x upsampling, maps low-dimensional features to high-dimensional features, allowing the network to maintain high-dimensional features while also paying attention to small target features; bilinear interpolation is used to achieve upsampling, which maintains the advantage of computational speed while fusing features.

2. A wild ginseng grading system based on an attention mechanism feature pyramid network, characterized in that: The system uses the method described in claim 1 to identify the grade of wild ginseng.

Images