Automatic grading and flaw detection method for bamboo shoots based on image processing

Abstract

The invention discloses an automatic bamboo shoot grading and flaw detection method based on image processing, and relates to the technical field of bamboo shoot food production, and the method comprises the following steps: designing a bamboo shoot image acquisition area, a bamboo shoot grading operation area, a grading operation manipulator and a bamboo shoot conveying belt; conveying to-be-detected bamboo shoots by using a bamboo shoot conveying belt, and collecting bamboo shoot images; the collected bamboo shoot images are preprocessed; performing defect detection on the preprocessed bamboo shoot image by using a pre-trained bamboo shoot defect detection model; according to the bamboo shoot defect detection result, utilizing a pre-trained bamboo shoot grading model to predict the grade of the bamboo shoots; and the grading operation manipulator determines the real-time position of the bamboo shoots according to the initial position data and the conveying speed data of the bamboo shoots, and stores the bamboo shoots to a bamboo shoot temporary storage area corresponding to the bamboo shoot grading operation area. Through automation and image processing technologies, high efficiency, high accuracy and high stability of bamboo shoot grading and flaw detection are realized, meanwhile, the production cost is reduced, the product quality is improved, and the market competitiveness is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of bamboo shoot food production, and in particular to an automatic bamboo shoot grading and defect detection method based on image processing. Background Art

[0002] Bamboo shoots, as a traditional Chinese delicacy, are loved by people for their unique flavor and nutritional value. Bamboo shoots are not only fragrant and crispy, but also rich in various nutrients, including protein, crude fiber, water, etc., which have a positive impact on human health. There are many types of bamboo shoots, and different types of bamboo shoots have different uses in food processing and cooking.

[0003] The bamboo shoot food processing industry has a long history in China, but with the development of society and the accelerated pace of people's lives, the demand for bamboo shoot food is also growing. As part of plant-based food, bamboo shoots have great market potential driven by the global trend of healthy eating. However, the current bamboo shoot food processing industry mainly faces the following challenges: After the bamboo shoot skin is peeled off, the traditional bamboo shoot grading and defect detection mainly rely on manual work, which is inefficient, labor-intensive, and the cost increases with the increase in labor costs; when manually detecting defects and grading bamboo shoots, due to strong subjectivity, the quality control is unstable, and it is difficult to ensure the consistency and reliability of the product; the existing bamboo shoot grading technology has defects, such as the inability to quickly and accurately grade and detect defects in bamboo shoots, resulting in low processing efficiency and difficulty in meeting the needs of large-scale production; although some automated equipment has been used for bamboo shoot grading, these devices are often complex in structure, high in cost, and cannot adapt well to bamboo shoots of different types and sizes, limiting their application in actual production. In summary, in order to improve the efficiency and quality of bamboo shoot food processing and reduce costs, it is urgently necessary to develop an automatic bamboo shoot grading and defect detection method based on image processing to solve the defects in the existing technology and improve the overall technical level of the bamboo shoot processing industry. Summary of the invention

[0004] The present invention provides a bamboo shoot automatic grading and defect detection method based on image processing, so as to realize efficient and accurate automatic grading and defect detection of bamboo shoots.

[0005] In order to solve the above problems, the technical solution adopted by the present invention is as follows: The present invention provides a method for automatic bamboo shoot grading and defect detection based on image processing, comprising: Step S1, designing a bamboo shoot image acquisition area, a bamboo shoot grading operation area with a bamboo shoot temporary storage area, a grading operation manipulator and a bamboo shoot conveyor belt, wherein there are multiple bamboo shoot grading operation areas, each corresponding to bamboo shoots of different grades; the bamboo shoot conveyor belt passes through the bamboo shoot image acquisition area and the bamboo shoot grading operation area; Step S2, using the bamboo shoot conveyor belt to convey the bamboo shoots to be detected, using an image acquisition device in the bamboo shoot image acquisition area to acquire bamboo shoot images, and saving initial position data of the bamboo shoots after image acquisition; Step S3, performing preprocessing including grayscale, geometric transformation, image enhancement and normalization on the collected bamboo shoots image; Step S4: using the pre-trained bamboo shoot defect detection model to perform defect detection on the pre-processed bamboo shoot image to obtain a bamboo shoot defect detection result; Step S5: predicting the grade of bamboo shoots using a pre-trained bamboo shoot grading model according to the bamboo shoot defect detection result; Step S6, the grading operation robot determines the real-time position of the bamboo shoots according to the initial position data of the bamboo shoots and the conveying speed data of the bamboo shoot conveyor belt, and removes the bamboo shoots from the bamboo shoot conveyor belt according to the grade of the bamboo shoots and stores them in the bamboo shoot temporary storage area corresponding to the bamboo shoot grading operation area.

[0006] Optionally, a bamboo shoot straightening device consisting of two boards arranged in an eight-shaped shape is provided at the starting position of the bamboo shoot conveyor belt, so as to make the bamboo shoots parallel to the conveying direction during the conveying process.

[0007] Optionally, each bamboo shoot grading operation area corresponds to a grading operation robot; the grading operation robot is a telescopic structure parallel to the belt surface of the bamboo shoot conveyor belt and perpendicular to the conveying direction, and its telescopic end is vertically arranged with a push plate that can cover the entire area length of the corresponding bamboo shoot grading operation area.

[0008] Optionally, the bamboo shoot image acquisition area has a bamboo shoot tumbling device for tumbling the bamboo shoots in the bamboo shoot image acquisition area, so that the bamboo shoot image acquired by the image acquisition device is a full-body image of the bamboo shoot.

[0009] Optionally, the architecture of the bamboo shoot defect detection model includes an input layer, a patch embedding module, a Swin transform block, a patch merging module, and a classification and regression head; the input layer of the bamboo shoot defect detection model is used to receive the bamboo shoot image data to be detected, and its patch embedding module is used to divide the input bamboo shoot image into a number of non-overlapping blocks, each block is regarded as a patch mark, and each block is converted into a feature vector through a linear embedding layer; the Swin transform block of the bamboo shoot defect detection model includes a regular window self-attention module and a shifted window self-attention module, wherein the regular window self-attention module is used to independently perform multi-head self-attention calculations in each window, and the shifted window self-attention module is used to realize information interaction between windows through a shifted window mechanism, thereby enhancing the representation ability of the model; the patch merging module of the bamboo shoot defect detection model is used to merge each group of adjacent patch marks into a new patch mark, and then reduce the dimension to 2D; the classification and regression head uses a fully connected layer and a convolutional layer to process the features output by the patch merging module to obtain a bamboo shoot defect detection result, including the category and bounding box coordinates of the bamboo shoot defect.

[0010] Optionally, the bamboo shoot defect detection model uses a pre-trained Swin Transformer model as the basic initialization model parameters during training, the loss function used in training is the Wise-IoU loss function, and the training strategy is to perform multi-scale training according to the model parameters and loss values, gradually increase the learning rate training, adjust the learning rate according to the cosine function, exponential moving average weight update and mixed precision training, and finally obtain the updated model parameters; after the pre-processed bamboo shoot image data used for training and the updated model parameters are input into the initialized model, the model parameters are updated through back propagation and optimizer to minimize the loss function, and then the model is evaluated and optimized.

[0011] Optionally, within a custom time period, the parameters of the bamboo shoot defect detection model are regularly updated by fine-tuning based on the latest collected bamboo shoot images, so that the model can adapt to new data features.

[0012] Optionally, the architecture of the bamboo shoot grading model includes an input layer, a feature processing layer, a classification layer and an output layer; the input layer of the bamboo shoot grading model is used to input the bamboo shoot defect detection results; the feature processing layer of the bamboo shoot grading model is used to perform linear transformation and nonlinear activation on the bamboo shoot defect detection results to extract higher-level feature representations and provide more abstract features for classification; the classification layer of the bamboo shoot grading model is used to classify the processed features using a fully connected layer, and map the output of the feature processing layer to the grade classification of bamboo shoots; the output layer of the bamboo shoot grading model is used to convert the output of the classification layer into a probability distribution using a softmax function, and predict the final grade of the bamboo shoots.

[0013] Optionally, during the training of the bamboo shoot grading model, the training data used is bamboo shoot image data and its corresponding defect detection results. It is necessary to manually annotate each bamboo shoot image according to the bamboo shoot grading standard to determine its grade and create a bamboo shoot defect detection result data set with grade labels.

[0014] Optionally, the user is allowed to customize the bamboo shoot grading standards of the bamboo shoot grading model according to needs.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1) This method realizes automatic image acquisition of bamboo shoots by linking the conveyor belt and the image acquisition device, replacing manual inspection. Combining the preprocessing and automatic operation of the defect detection model and the grading model, the system can realize the rapid processing of a large number of bamboo shoots, greatly improving the grading and detection efficiency.

[0016] 2) Using pre-trained bamboo shoot defect detection and grading models, the defect types and degrees of bamboo shoots are accurately identified through image analysis and machine learning algorithms, which reduces subjective errors in manual grading and makes the grading results more accurate and reliable.

[0017] 3) The present invention realizes automatic grading and storage through a grading operation robot, which reduces the labor burden of manual grading and reduces dependence on manual labor, thereby saving human resources and improving overall economic benefits.

[0018] 4) This method can ensure the consistency of the inspection standards for each bamboo shoot through image analysis, realize the standardization and regularization of grading, and improve the market competitiveness of bamboo shoot products and the consistency of their quality.

[0019] 5) When performing bamboo shoot defect detection, the bamboo shoot image is segmented into non-overlapping blocks through the patch embedding module, and each block is converted into a feature vector, which helps to refine feature extraction and improve the sensitivity and accuracy of defect detection; the introduction of the Swin transform block, through the combination of the regular window self-attention module and the shifted window self-attention module, enables the model to effectively capture local and global information, thereby improving the representation ability of bamboo shoot defect features; the patch merging module can integrate feature information by merging adjacent patches, so that the final classification and regression head can more accurately output the category and bounding box coordinates of the bamboo shoot defect, and achieve high-quality detection results. The bamboo shoot defect detection model adopted combines strategies such as multi-scale training, gradually increasing learning rate training, adjusting the learning rate according to the cosine function, exponential moving average weight update, and mixed precision training, making the model more stable and efficient during the training process, and finally obtaining more robust model parameters, further improving the reliability and accuracy of defect detection.

[0020] 6) When grading bamboo shoots, the feature processing layer of the bamboo shoot grading model effectively extracts high-level features of the bamboo shoot defect detection results through linear transformation and nonlinear activation, so that the model can capture more complex feature relationships and provide more accurate feature representation for subsequent classification tasks. The classification layer of the bamboo shoot grading model effectively maps the processed features through the fully connected layer to achieve accurate classification of bamboo shoot grades. This structure improves the model's ability to distinguish bamboo shoots of different grades and ensures the accuracy of the grading results. The output layer uses the softmax function to convert the results of the classification layer into probability distributions, which is convenient for predicting the final grade of the bamboo shoots. During training, the bamboo shoot grading model constructs a bamboo shoot defect detection result dataset with grade labels, so that the model can better adapt to the characteristics of various bamboo shoots during training, enhances the generalization ability of the model, and improves its performance in practical applications.

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the embodiments of the present invention are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a method for automatic bamboo shoot grading and defect detection in an embodiment; Figure 2 3D schematic diagram of the system for automatic bamboo shoot grading and defect detection in the embodiment; Figure 3 3D schematic diagram of the system for automatic bamboo shoot grading and defect detection in the embodiment; In the figure: 1. Bamboo shoot conveyor belt; 2. Image acquisition device; 3. Bamboo shoot straightening device; 4. Grading operation robot; 5. Push plate. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0025] Please refer to Figure 1 This embodiment provides a method for automatic bamboo shoot grading and defect detection based on image processing, which is as follows: Step S1, designing a bamboo shoot image acquisition area, a bamboo shoot grading operation area with a bamboo shoot temporary storage area, a grading operation manipulator 4 and a bamboo shoot conveyor belt 1, there are multiple bamboo shoot grading operation areas, and each corresponds to a bamboo shoot of different grades, and the bamboo shoot conveyor belt 1 passes through the bamboo shoot image acquisition area and the bamboo shoot grading operation area, see Figure 1 and Figure 2 shown.

[0026] The bamboo shoot image acquisition area is the first key part of the system, responsible for the acquisition of bamboo shoot images. A high-resolution image acquisition device 2, such as an industrial camera or scanner, is installed in this area to capture detailed images of bamboo shoots. The image acquisition device 2 should have automatic focus and exposure adjustment functions to adapt to bamboo shoots of different sizes and shapes to ensure image clarity. The acquisition area can also be equipped with an appropriate lighting system to provide uniform lighting conditions and reduce the impact of shadows and reflections on image quality.

[0027] Each bamboo shoot grading operation area is equipped with a bamboo shoot temporary storage area for temporary storage of bamboo shoots that have been graded and are waiting for further processing. The design of the temporary storage area should take into account the protection of the bamboo shoots to avoid damage during the grading process, so protective measures such as soft pads or partitions can be designed.

[0028] The number of bamboo shoot grading operation areas corresponds to the bamboo shoot grading standards. Each bamboo shoot grading operation area corresponds to bamboo shoots of different grades. These areas are designed according to the size, shape, appearance and other characteristics of the bamboo shoots to meet the grading needs of bamboo shoots of different grades. Each operation area is equipped with a robot to transfer bamboo shoots of determined grades from the conveyor belt to the temporary storage area.

[0029] like Figure 1 As shown, in this embodiment, a bamboo shoot straightening device 3 composed of two boards arranged in an eight-shaped layout is set at the starting position of the bamboo shoot conveyor belt 1, which is used to make the bamboo shoots parallel to the conveying direction during the conveying process. The board is made of wear-resistant and easy-to-clean materials to ensure durability and hygiene in long-term use, which meets the requirements of the food processing industry. When the bamboo shoots pass through the straightening device, the eight-shaped design can effectively guide the bamboo shoots to the central position of the conveyor belt, making it parallel to the conveying direction, which can reduce the mutual collision of the bamboo shoots during the conveying process and ensure that the bamboo shoots are neatly arranged on the conveyor belt. The straightening device can effectively prevent the bamboo shoots from being deflected during the conveying process, ensure that the bamboo shoots are in a stable position when they are in the image acquisition area, and improve the accuracy and clarity of image acquisition. In the process of the bamboo shoots being pushed by the conveyor belt, the design of the straightening device uses gravity and the tilt angle to cause the bamboo shoots to naturally move closer to the boards on both sides, thereby achieving straightening. Under the guidance of the figure-eight board, the bamboo shoots are gradually adjusted to the center position to ensure their parallel movement on the conveyor belt. The straightened bamboo shoots smoothly enter the image acquisition area, providing a stable foundation for subsequent image capture and ensuring that each bamboo shoot can be accurately collected and analyzed.

[0030] In this embodiment, each bamboo shoot grading operation area corresponds to a grading operation manipulator 4; the grading operation manipulator 4 is a telescopic structure parallel to the belt surface of the bamboo shoot conveyor belt 1 and perpendicular to the conveying direction, and its telescopic end is vertically arranged to cover the push plate 5 of the entire length of the corresponding bamboo shoot grading operation area. The telescopic design can adjust the length as needed, so that it can flexibly adapt to the layout of bamboo shoots and grading operation areas of different sizes and heights. The push plate 5 equipped at the telescopic end can cover the entire area of ​​the corresponding bamboo shoot grading operation area, ensuring that during the operation, the manipulator can cover the position of each bamboo shoot, which is convenient for grading and storage. Since the manipulator can quickly retract and cover the entire grading operation area, its efficiency is significantly improved, reducing the residence time of the bamboo shoots on the conveyor belt and improving the speed of the overall grading operation. In addition, a large manipulator with multiple degrees of freedom can also be configured to replace the grading operation manipulator 4 of this embodiment, but the cost is higher, and the advantage is that only one needs to be arranged.

[0031] Since bamboo shoots are cylindrical in structure, generally speaking, Figure 1 , Figure 2 In the layout shown, the image acquisition device 2 can acquire most of the area of ​​the bamboo shoots. Through multiple manual observation tests, we found that the acquired image can fully identify the level of the image. Of course, for greater accuracy, a bamboo shoot rolling device can also be designed in the bamboo shoot image acquisition area to roll the bamboo shoots in the bamboo shoot image acquisition area, so that the bamboo shoot image acquired by the image acquisition device 2 is a full-body image of the bamboo shoot.

[0032] Step S2, using the bamboo shoot conveyor belt 1 to convey the bamboo shoots to be detected, using the image acquisition device 2 to acquire the bamboo shoot images in the bamboo shoot image acquisition area, and saving the initial position data of the bamboo shoots after the image acquisition is completed.

[0033] Before the detection process begins, the bamboo shoot conveyor belt 1 is started to operate normally to ensure that the bamboo shoots to be detected on the belt can continue to move. According to the working efficiency of the entire detection system and the response speed of the image acquisition device, an appropriate conveying speed is set to ensure that the bamboo shoots stay in the image acquisition area for a sufficient time to obtain a clear image.

[0034] Before the bamboo shoots enter the conveyor belt, the straightening device adjusts the bamboo shoots to a state parallel to the conveying direction to ensure stability during the subsequent image acquisition process. Position sensors can be installed on the conveyor belt to monitor the position of the bamboo shoots on the conveyor belt in real time so as to accurately obtain their position coordinates.

[0035] When the bamboo shoots enter the image acquisition area, the system automatically activates the image acquisition device 2 to prepare to capture the image of the bamboo shoots. When the bamboo shoots pass through the image acquisition area, the image acquisition device 2 automatically captures their images to ensure that the bamboo shoots are in a clear and complete state during the acquisition process. While capturing the image, the system will record the initial position data of the bamboo shoots, including their position coordinates on the conveyor belt (such as X-axis and Y-axis coordinates). The captured image data and the initial position data of the bamboo shoots are stored in the database together to provide necessary information for subsequent defect detection and grading.

[0036] The image acquisition and position data acquisition process is observed in real time through the monitoring interface to ensure stable operation of the equipment. After the image acquisition is completed, the data is ready to be transmitted to the subsequent processing module for defect detection and classification analysis.

[0037] Step S3: performing preprocessing including grayscale conversion, geometric transformation, image enhancement and normalization on the collected bamboo shoot images.

[0038] Grayscale processing uses the weighted average method to convert the pixel values ​​of the RGB color channels into a single grayscale value, converting the color bamboo shoot image into a grayscale image, reducing the complexity of the image data, retaining the main features, and improving the efficiency of subsequent processing.

[0039] Geometric transformation is to change the size of the image through a scaling function to adapt to the model input requirements, rotate the image at random angles to ensure that the model can accurately identify bamboo shoots at different angles, and flip the image horizontally or vertically to increase data diversity. Scaling, rotating and flipping the image enhances the model's robustness to bamboo shoots at different viewing angles and sizes.

[0040] Image enhancement is based on histogram equalization to adjust image brightness and contrast, enhance details, reduce image noise based on Gaussian filtering to ensure clear details, and use edge detection algorithms (such as Sobel operator) to enhance the edge information of the image and highlight the shape and structural features of bamboo shoots. Image enhancement improves the visual quality of the image, enhances important features, and improves the accuracy of subsequent defect detection. Normalization scales the pixel values ​​of the image to a uniform range (usually [0, 1] or [-1, 1]) to improve the training efficiency and stability of the model.

[0041] Step S4: Use the pre-trained bamboo shoot defect detection model to perform defect detection on the preprocessed bamboo shoot image to obtain a bamboo shoot defect detection result.

[0042] The architecture of the bamboo shoot defect detection model includes an input layer, a patch embedding module, a Swin transformation block, a patch merging module, and classification and regression heads.

[0043] The input layer is used to receive the bamboo shoot image data to be detected. The image dimension is ,in is the image height, is the image width, is the number of color channels of the image.

[0044] The patch embedding module is used to segment the input bamboo shoot image into N fixed-size non-overlapping blocks, N is the total number of blocks, and each block is regarded as a patch marker. Each block is converted into a feature vector through a linear embedding layer, and the dimension is Transformed to D-dimensional feature space, where P is the size of the patch and D is the embedding dimension. After being processed by the patch embedding module, the output is an N×D feature sequence, which is input into the Swin transform block for processing.

[0045] The Swin transformation block consists of two parts, namely the regular window self-attention module (W-MSA) and the shifted window self-attention module (SW-MSA). The regular window self-attention module is used to perform multi-head self-attention calculations independently in each window. The shifted window self-attention module is used to achieve information interaction between windows through the shift window mechanism and enhance the representation ability of the model. The feature sequence processed by the multi-head self-attention mechanism still has a dimension of N×D, and the data is input into the patch merging module for processing.

[0046] The patch merging module is used to downsample, reduce the resolution, adjust the number of channels, and thus form a hierarchical design, while also saving a certain amount of computation. The patch merging module merges each group of 2×2 adjacent patch markers into a new patch marker, increasing the number of channels to 4D, and then reduces the dimension to 2D through a linear layer. After being processed by the patch merging module, the output is of dimension The feature sequence is N / 4 (where N / 4 is because the spatial dimension is halved), which is used to input the classification and regression heads.

[0047] The classification and regression head uses the detection head to predict the bounding box and category of the bamboo shoot defect. Specifically, the fully connected layer (FC) and convolutional layer are used to process the features to obtain the bamboo shoot defect detection results, including the category and bounding box coordinates of the bamboo shoot defect.

[0048] When training the bamboo shoot defect detection model, the original bamboo shoot image data used for training is first preprocessed, including grayscale, geometric transformation, image enhancement, and normalization. Then, the pre-trained Swin Transformer model is used as the basis to initialize the model parameters, and then the initialized model can be trained.

[0049] The loss function used in training is the Wise-IoU (wise intersection-over-union) loss function. in, Wise-IoU loss is used to measure the difference between the bounding box predicted by the model and the true bounding box; is the bounding box predicted by the model, is the ground-truth bounding box.

[0050] The training strategy of the bamboo shoot defect detection model is to perform multi-scale training (training the model at different scales) based on the model parameters and loss values, gradually increase the learning rate training, adjust the learning rate according to the cosine function, EMA (exponential moving average) weight update and mixed precision training (using FP16 and FP32 combined training), and finally obtain the updated model parameters. Among them, FP16 refers to 16-bit half-precision floating point numbers. FP32 refers to 32-bit single-precision floating point numbers.

[0051] The formula for adjusting the learning rate according to the cosine function is: in, is the learning rate at step t, is the initial learning rate, t is the current number of training steps, and T is the total number of training steps.

[0052] The EMA weight update formula is: in, is the EMA parameter at step t, is the smoothing factor, is the current parameter.

[0053] After the preprocessed bamboo shoot image data and updated model parameters for training are input into the initialized model, the model parameters are updated through back propagation and optimizer to minimize the loss function. Model evaluation and optimization are then performed. In addition, the model can be further trained on specific tasks to improve the accuracy and robustness of the model. During the model evaluation and optimization process, the present invention mainly focuses on indicators such as precision, recall rate and average precision, and adjusts the model structure or training strategy according to the evaluation results.

[0054] Step S5: predict the grade of the bamboo shoots using the pre-trained bamboo shoot grading model according to the bamboo shoot defect detection results.

[0055] The architecture of the bamboo shoot grading model includes an input layer, a feature processing layer, a classification layer, and an output layer. The input layer is used to input the bamboo shoot defect detection results obtained in step S4 (as feature vectors), and the feature processing layer is used to perform linear transformation and nonlinear activation on the bamboo shoot defect detection results to extract higher-level feature representations and provide more abstract features for classification. The processed feature vectors ,in and are the weight and bias of the layer respectively, is the feature vector received by the input layer, and ReLU is the activation function. The classification layer is used to classify the processed features using the fully connected layer, mapping the output of the feature processing layer to the grade classification of bamboo shoots. The output of the classification layer ,in and are the weights and biases of this layer, is the output of the classification layer, is the processed feature vector. Its output layer is used to convert the output of the classification layer into a probability distribution using the softmax function to predict the final grade of bamboo shoots. Each grade corresponds to a probability value. The predicted bamboo shoot grade probability distribution is .

[0056] During the training process of the bamboo shoot grading model, the training data used are bamboo shoot image data and its corresponding defect detection results. It is necessary to manually annotate each bamboo shoot image according to the bamboo shoot grading standard to determine its grade and create a bamboo shoot defect detection result dataset with grade labels for training the bamboo shoot grading model.

[0057] In this embodiment, the bamboo shoot grading standards are shown in Table 1.

[0058] Table 1 grade describe Excellent No obvious defects, uniform length, bright color, tender texture, and free of diseases and insect pests. Good Minor flaws, slightly uneven length, good color, tender texture, and no serious diseases or pests. Secondary There are many flaws, large length differences, dull color, older texture, and possible pests and diseases. After the preparation of training data is completed, features that are helpful for grading can be extracted from the labeled bamboo shoot defect detection results through feature engineering to convert the original data into a format that the model can process, improve the performance and accuracy of the model, and then obtain a characterized data set. For the bamboo shoot grading model of the above architecture, we use the cross entropy loss function calculator training loss to measure the difference between the grade probability distribution predicted by the model and the actual label, guide the direction of model training, and make the grade predicted by the model as close to the actual label as possible.

[0059] The cross entropy loss function is: in, L is the cross entropy loss, is the one-hot encoding of the true label, is the probability distribution predicted by the model and C is the number of categories.

[0060] In the training of the bamboo shoot grading model, the Adam optimizer is selected to update the model weights to minimize the loss. The value of the loss function is reduced by adjusting the model parameters to improve the prediction accuracy of the model. The bamboo shoot grading model is trained using the bamboo shoot defect detection result dataset, and the model parameters are adjusted through the back propagation algorithm so that the bamboo shoot grading model learns how to predict the grade of bamboo shoots based on the input features.

[0061] Step S6, the grading operation robot 4 determines the real-time position of the bamboo shoots according to the initial position data of the bamboo shoots and the conveying speed data of the bamboo shoot conveyor belt 1, and removes the bamboo shoots from the bamboo shoot conveyor belt 1 according to the grade of the bamboo shoots and stores them in the bamboo shoot temporary storage area corresponding to the bamboo shoot grading operation area.

[0062] In this embodiment, when the graded bamboo shoots enter the bamboo shoot grading operation area of ​​the corresponding level, the grading operation robot 4 in this area pushes the bamboo shoots to the bamboo shoot temporary storage area of ​​the bamboo shoot grading operation area on one side of the conveyor belt, and then the grading operation robot 4 quickly resets to wait for the next operation.

[0063] In an optional embodiment of the present invention, we can also design to regularly and adaptively update the parameters of the bamboo shoot defect detection model using the latest collected bamboo shoot images within a custom time period. The specific process includes: The latest bamboo shoot images are preprocessed, including grayscale, geometric transformation, image enhancement and normalization. Then, each bamboo shoot image is labeled according to the bamboo shoot grading standard to determine its defect type and location. Then, fine-tuning is adopted based on the labeled bamboo shoot image dataset to update only the parameters of the bamboo shoot defect detection model without changing the model architecture, so that the model can adapt to the new data features and improve the model's ability to recognize new defect types.

[0064] Fine-tune on the new dataset and update the model parameter formula as follows: in, are the updated model parameters, are the old model parameters, is the learning rate, is the loss function About parameters gradient.

[0065] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for automatic bamboo shoot grading and defect detection based on image processing, characterized in that: include: Step S1, designing a bamboo shoot image acquisition area, a bamboo shoot grading operation area with a bamboo shoot temporary storage area, a grading operation manipulator and a bamboo shoot conveyor belt, wherein there are multiple bamboo shoot grading operation areas, each corresponding to bamboo shoots of different grades; the bamboo shoot conveyor belt passes through the bamboo shoot image acquisition area and the bamboo shoot grading operation area; Step S2, using the bamboo shoot conveyor belt to convey the bamboo shoots to be detected, using an image acquisition device in the bamboo shoot image acquisition area to acquire bamboo shoot images, and saving initial position data of the bamboo shoots after image acquisition; Step S3, performing preprocessing including grayscale, geometric transformation, image enhancement and normalization on the collected bamboo shoots image; Step S4: using the pre-trained bamboo shoot defect detection model to perform defect detection on the pre-processed bamboo shoot image to obtain a bamboo shoot defect detection result; Step S5: predicting the grade of bamboo shoots using a pre-trained bamboo shoot grading model according to the bamboo shoot defect detection result; Step S6, the grading operation robot determines the real-time position of the bamboo shoots according to the initial position data of the bamboo shoots and the conveying speed data of the bamboo shoot conveyor belt, and removes the bamboo shoots from the bamboo shoot conveyor belt according to the grade of the bamboo shoots and stores them in the bamboo shoot temporary storage area corresponding to the bamboo shoot grading operation area.

2. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 1, characterized in that: A bamboo shoot straightening device consisting of two boards arranged in an eight-shaped pattern is arranged at the starting position of the bamboo shoot conveyor belt, which is used to make the bamboo shoots parallel to the conveying direction during the conveying process.

3. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 1, characterized in that: Each bamboo shoot grading operation area corresponds to a grading operation robot; the grading operation robot is a telescopic structure parallel to the belt surface of the bamboo shoot conveyor belt and perpendicular to the conveying direction, and its telescopic end is vertically arranged with a push plate that can cover the entire area length of the corresponding bamboo shoot grading operation area.

4. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 1, characterized in that: The bamboo shoot image acquisition area is provided with a bamboo shoot rolling device, which is used to roll the bamboo shoots in the bamboo shoot image acquisition area, so that the bamboo shoot image acquired by the image acquisition device is a full-body image of the bamboo shoot.

5. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 1, characterized in that: The architecture of the bamboo shoot defect detection model includes an input layer, a patch embedding module, a Swin transformation block, a patch merging module, and a classification and regression head; the input layer of the bamboo shoot defect detection model is used to receive the bamboo shoot image data to be detected, and its patch embedding module is used to divide the input bamboo shoot image into a number of non-overlapping blocks, each block is regarded as a patch mark, and each block is converted into a feature vector through a linear embedding layer; the Swin transformation block of the bamboo shoot defect detection model includes a regular window self-attention module and a shifted window self-attention module, wherein the regular window self-attention module is used to independently perform multi-head self-attention calculations in each window, and the shifted window self-attention module is used to realize information interaction between windows through a shifted window mechanism, thereby enhancing the representation ability of the model; the patch merging module of the bamboo shoot defect detection model is used to merge each group of adjacent patch marks into a new patch mark, and then reduce the dimension to 2D; the classification and regression head uses a fully connected layer and a convolutional layer to process the features output by the patch merging module to obtain a bamboo shoot defect detection result, including the category and bounding box coordinates of the bamboo shoot defect.

6. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 5, characterized in that: The bamboo shoot defect detection model uses a pre-trained Swin Transformer model as the basic initialization model parameters during training. The loss function used in training is the Wise-IoU loss function. The training strategy is to perform multi-scale training according to the model parameters and loss values, gradually increase the learning rate training, adjust the learning rate according to the cosine function, update the exponential moving average weights and perform mixed precision training, and finally obtain the updated model parameters; after the pre-processed bamboo shoot image data used for training and the updated model parameters are input into the initialized model, the model parameters are updated through back propagation and optimizer to minimize the loss function, and then the model is evaluated and optimized.

7. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 1, characterized in that: Within a custom time period, the parameters of the bamboo shoot defect detection model are regularly updated by fine-tuning based on the latest collected bamboo shoot images, so that the model can adapt to new data features.

8. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 1, characterized in that: The architecture of the bamboo shoot grading model includes an input layer, a feature processing layer, a classification layer and an output layer; the input layer of the bamboo shoot grading model is used to input the bamboo shoot defect detection results; the feature processing layer of the bamboo shoot grading model is used to perform linear transformation and nonlinear activation on the bamboo shoot defect detection results to extract higher-level feature representations and provide more abstract features for classification; the classification layer of the bamboo shoot grading model is used to classify the processed features using a fully connected layer, and map the output of the feature processing layer to the grade classification of bamboo shoots; the output layer of the bamboo shoot grading model is used to convert the output of the classification layer into a probability distribution using a softmax function to predict the final grade of the bamboo shoots.

9. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 8, characterized in that: During the training process of the bamboo shoot grading model, the training data used are bamboo shoot image data and its corresponding defect detection results. It is necessary to manually annotate each bamboo shoot image according to the bamboo shoot grading standard, determine its grade, and create a bamboo shoot defect detection result dataset with grade labels.

10. The method for automatic bamboo shoot grading and defect detection based on image processing according to claim 1, characterized in that: Allow users to customize the bamboo shoot grading standards of the bamboo shoot grading model according to their needs.

Images