Fresh cut rose flower openness grading method
Through deep learning object detection algorithm and neural network model, combined with SE and EMA attention mechanism, the inefficiency and accuracy of manual judgment in the grading of rose fresh cut flowers is solved, and the rapid and accurate openness recognition and classification are achieved, and the adaptability of the rapid and accurate openness recognition and classification are achieved. Feature inconsistency improves recognition efficiency and accuracy.
Patent Information
- Application Number
- CN202411165582.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-29
AI Technical Summary
The existing method of grading the blooming degree of rose fresh cut flowers relies on manual judgment, which has inefficient efficiency and visual fatigue of operators, and the difference in color between winter and summer affects the difficulty of judgment, and blurred boundaries make it difficult to accurately grading.
Using deep learning object detection algorithm, a neural network model is constructed by acquiring and preprocessing cut flower images, combining SE channel attention mechanism and EMA attention mechanism to perform feature extraction and classification, identify the openness species of freshly cut rose flowers and return to the target Bounding box, providing accurate category information.
It improves the accuracy and efficiency of identifying the openness of fresh cut roses, reduces the error caused by manual intervention, and achieves rapid and accurate openness positioning and classification, adapts to the inconsistency of rose characteristics, and improves corporate profits.
Smart Images

Figure CN119048739A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a method for grading the degree of openness of fresh cut rose flowers. Background Art
[0002] Openness quality grading is an important link in the production of fresh cut roses. The degree of openness of fresh cut roses will determine their storage and transportation value. Lower openness has good potential value, while higher openness has the opposite.
[0003] The current packaging process for fresh cut roses in China is: 15-20 roses are packed in a group, tightly packed with corrugated paper. In the wholesale process of fresh cut flowers, downstream sellers set prices based on the flowers with the highest degree of openness in each group of fresh cut roses. Therefore, accurate judgment of the degree of openness will have a huge impact on corporate profits.
[0004] However, the boundaries between different degrees of openness are fuzzy and difficult to distinguish, and there is a slight difference in the degree of openness between winter and summer, and petals of different colors interfere with judgment. At the same time, the current degree of openness of cut flowers is heavily dependent on manual labor, which has problems such as low efficiency and inaccurate grading caused by operator visual fatigue. Summary of the invention
[0005] In order to solve or partially solve the problems existing in the related art, the present application provides a rose fresh cut flower opening degree grading method, which identifies the rose fresh cut flower opening degree type and regresses the target's bounding box based on a deep learning target detection algorithm, providing category information for rose fresh cut flower opening degree determination.
[0006] The first aspect of the present application provides a method for grading the degree of openness of fresh cut roses, comprising the following steps:
[0007] First, obtain cut flower images and generate data sets after preprocessing;
[0008] Secondly, the dataset is annotated and visualized, and data augmentation is performed on the deformed category images in the dataset;
[0009] Among them, labeling the data set and visual analysis include:
[0010] The images in the dataset are labeled with openness categories and bounding boxes, where the bounding boxes are true boxes;
[0011] Visually analyze the annotated images to obtain deformed category images, where the number of true boxes is less than half of the average of all category true boxes.
[0012] Among them, data expansion of deformed category images of the dataset includes:
[0013] Get the original image of the deformed category image;
[0014] Select the original image or cut out the real frame of the original image;
[0015] The transformed image is adjusted by adding noise, random rotation, random flipping, and adding image distortion;
[0016] Add the transformed images to the dataset.
[0017] Secondly, a neural network model for feature extraction and classification is constructed and trained and optimized. The data set is input into the optimization model to obtain the prediction results output by the optimization model. The optimization model uses Mobilenetv3 as the feature extraction module, optimizes the feature extraction, and introduces the SE channel attention mechanism and the EMA attention mechanism.
[0018] Among them, training and optimizing the neural network model includes:
[0019] Input the training set to the neural network model;
[0020] After training, cross validation is performed, and after calculating the accuracy index, a learning rate adjustment strategy is adopted and the network is fine-tuned;
[0021] When the model is trained to the set number of iterations, the training is stopped and the model and training parameters are saved.
[0022] Thirdly, determine the confidence and bounding box in the prediction result, sort the bounding boxes based on the confidence, and select the bounding box with the highest confidence as the reference box;
[0023] Finally, the bounding box is filtered by the reference box and the recognition result is output.
[0024] Among them, filtering the bounding box based on the reference box includes:
[0025] Calculate the intersection-over-union ratio of the base box with all other bounding boxes in the list;
[0026] If the overlap index between the current bounding box and the reference box exceeds a preset overlap threshold, the current bounding box is suppressed and removed;
[0027] Repeatedly select the new highest confidence ground truth box and perform suppression until all bounding boxes are calculated.
[0028] The output recognition results include:
[0029] Calculate the category confidence of the bounding box for five openness categories: first degree, second degree, third degree, fourth degree, and deformity;
[0030] The category confidences are sorted and the category with the highest confidence is marked as the openness level of the identification target.
[0031] Among them, the neural network model includes:
[0032] The feature extraction module is used to extract the features of the data set and perform convolution, batch normalization and depth-separable convolution on the features;
[0033] Neck module, including upsampling part and downsampling part. The upsampling part is used to obtain feature information of different scales, and the downsampling part is used to fuse feature information and input feature maps into the corresponding detection head to predict the target;
[0034] The prediction module includes three detection heads. The detection head contains two branches, which are used to calculate the position loss and the category loss.
[0035] Among them, the position loss includes intelligent intersection-over-union loss and local feature loss. The intelligent intersection-over-union loss includes first-level WIoU loss, second-level WIoU loss and third-level WIoU loss. The first-level WIoU loss introduces the normalized distance gain construction between the center point of the predicted box and the real box on the basis of the intersection-over-union loss. The second-level WIoU loss introduces the outlier construction on the basis of the first-level WIoU loss. The third-level WIoU loss introduces the hyperparameter construction on the basis of the second-level WIoU loss.
[0036] in,
[0037] Level WIoU loss L WIoUv1 , Secondary WIoU loss L WIoUv2 And the three-level WIoU loss L WIoUv3 It is expressed as:
[0038] L WIoUv1 =R WIoU L IoU
[0039] L WIoUv2 =(β) γ L WIoUv1
[0040]
[0041] In the formula, R WIoU is the normalized distance gain between the predicted box and the center point of the real box, L Io U is the intersection-over-union loss, β is the outlier degree describing the quality of the anchor box, and σ and α are two hyperparameters for adjusting the β gain.
[0042] Among them, the EMA attention mechanism performs parallel pooling operations on feature maps in wide and high dimensions, expands the feature space through parallel 3×3 convolutions, and aggregates the three branches across space after processing.
[0043] The technical solution provided by this application may have the following beneficial effects:
[0044] The present application provides a rose fresh cut flower opening degree grading method, which can better adapt to the inconsistency of rose characteristics in the rose fresh cut flower opening degree grading, improve the recognition accuracy of the network, so as to quickly and accurately locate and classify the rose fresh cut flower opening degree identification parts, automatically realize the rose fresh cut flower opening degree grading, reduce the recognition error caused by manual intervention, improve the openness recognition efficiency and accuracy, and thus increase corporate profits.
[0045] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.
[0047] Figure 1 The figure is a flow chart of the method for grading the degree of openness of fresh cut rose flowers;
[0048] Figure 2 A feature extraction and classification model diagram constructed in the embodiment;
[0049] Figure 3 It is a comparison diagram of Loss curves optimized in the embodiment;
[0050] Figure 4 Schematic diagram of the classification results obtained in the embodiment. DETAILED DESCRIPTION
[0051] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.
[0052] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0053] In the description of the present application, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.
[0054] Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0055] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0056] like Figure 1 The rose fresh cut flower opening degree grading method shown comprises the following steps:
[0057] S1. Obtain cut flower images and generate data sets after preprocessing.
[0058] S11. Data collection:
[0059] An industrial camera was used to capture images of fresh cut roses of the Pink Snow Mountain variety, and 2031 images of fresh cut roses were collected.
[0060] S12. Image preprocessing:
[0061] After analyzing the image data collected by S11, data expansion is performed on categories with fewer occurrences, and a small amount of Gaussian noise is added to them to improve the network's reasoning and generalization capabilities, which are used as classification data sets.
[0062] S13, data set division;
[0063] The enhanced images in S12 were used to create a YOLO format dataset, and the rose cut flower categories and bounding boxes in the images were annotated. The annotated location was the area from the unexpanded petals to the third outer petals. The annotated dataset was then randomly divided into an untrained set and a test set in a ratio of 8:2; the training set contained 1,625 images, and the test set contained 406 images.
[0064] S2. Label the dataset and conduct visual analysis, and perform data expansion on the deformed category images in the dataset.
[0065] Visual analysis is performed on the labeled data set. Each image in the data set has an annotated box. The training set contains 1,625 real boxes and the test set contains 406 real boxes. Data expansion is performed on the deformed categories. The real boxes of the aforementioned images are cut out or the entire image is selected. Then, adjustments are made by adding noise, random rotation, random flipping, adding image distortion, etc., and the converted results are added to the data set to complete data expansion.
[0066] The size ratio of the real frame is analyzed. The real frame is equivalent to about 12%-30% of the original image area. According to the definition of relative size, when the length and width of the target real frame are less than 10% of the length and width of the original image, that is, the area of the target real frame is less than 1% of the original image, it is called a small target. The analysis results show that the number of medium-sized ratios accounts for the vast majority, followed by larger ratios, that is, the detection task is mainly for medium and large targets.
[0067] S3. Build a neural network model for feature extraction and classification and train and optimize it. Input the data set into the optimization model to obtain the prediction results output by the optimization model.
[0068] Optimize the feature extraction part of the YOLOv8 network and build a neural network model for feature extraction and classification, such as Figure 2 As shown in the figure, the MoblilenetV2 feature extraction module replaces the original Darknet53 backbone to form a new feature extraction module. The input feature map is downsampled in three parts in the MoblilenetV3 backbone:
[0069] First, in the first part, a convolutional layer with a size of 3×3, a stride of 2, and a padding of 1 is used to extract image features and reduce the image volume. The convolutional image is scaled to the same scale through a batch normalization layer (BatchNormalization).
[0070] Then, in the second part, the feature map is processed by multiple depthwise separable convolutions. Depthwise separable convolutions include depthwise convolutions and pointwise convolutions. Depthwise convolutions use 1×1 convolution kernels to expand the number of channels, while pointwise convolutions use 3×3 convolution kernels to achieve feature extraction. This method helps to refine feature extraction while reducing the number of parameters.
[0071] MoblilenetV2 also introduces the SE channel attention mechanism to enhance the attention to different feature channels. The SE module mainly includes two operations, Squeeze and Excitation, which can model the correlation between feature channels and strengthen important features. The Squeeze operation uses G1obal Average Pooling to encode the entire spatial feature on a channel into a global feature; the Excitation operation uses a sigmoid gate mechanism to capture the relationship between channels. Two fully connected layers are used. The first fully connected layer reduces the dimension, then activates through ReLu, and then restores the dimension through the second fully connected layer. Then sigmoid activation is used, and finally the activation value of each channel learned is multiplied by the original feature. The entire SE operation can be regarded as the weight coefficient of each channel of the learning feature.
[0072] The Neck part of the model consists of two parts: upsampling and downsampling. The upsampling part consists of two "upsampling modules-splicing modules-c2f modules". The function of the upsampling module is to increase the dimension of the feature map, and then splice the feature map with the p4 and p3 feature maps of the trunk in the splicing module to obtain feature information of different scales. The EMA attention mechanism is introduced after the upsampling module. This mechanism performs parallel pooling operations on the feature map in the wide and high dimensions respectively, and expands the feature space through parallel 3×3 convolutions. After a series of processing, the three branches are aggregated across space. This attention mechanism will help the model better integrate the high-level semantic features of flowers. The downsampling part consists of multiple 3×3 convolutions, splicing modules, and c2f modules. The purpose of downsampling is to integrate features of different scales and input feature maps of different scales into detection heads of different scales such as large, medium, and samll to predict targets.
[0073] The prediction part of the model consists of three detection heads, each of which is further divided into two branches, each of which consists of a CBL+CBL+Conv module. One branch is used to calculate the position loss of the predicted box and the true box, which is a combination of the full generalized intersection-over-union (CIoU) loss and the local feature loss (DFL) loss; the other branch is used to calculate the prediction box category loss (BCE Loss). The CIoU loss is replaced by the smart intersection-over-union loss Wise-IoU loss. Wise-IoU adds a multi-layer gain based on the intersection-over-union loss (IoU), by adding the normalized distance (R) between the center point of the predicted box and the true box. WIoU ) gain, constructing the first-level WIoU loss (L WIoUv1 ). When the predicted box and the true box overlap well, R WIoUOn the basis of IoU loss, the gradient gain of the predicted box will be further reduced to reduce the network's attention to the predicted box with good overlap; when the predicted box and the real box have poor overlap, R WIoU The gradient gain of the predicted box will be increased. On this basis, in order to enable the model to allocate smaller gradient gains to high-quality targets with small outliers, so that the model can focus on difficult targets and dynamically adjust the loss, a secondary WIoU loss L is constructed. WIoUv2 And the three-level WIoU loss L WIoUv3 β is used to describe the outlier degree of the anchor box quality, and the gain of β is adjusted by two hyperparameters σ and α.
[0074] L WIoUv1 =R WIoU L IoU
[0075] L WIoUv2 =(β) γ L WIoUv1
[0076]
[0077] Use the model trained on the coco128 dataset for transfer training, train the model input training set, perform cross-validation after each round of training, and calculate accuracy indicators such as loss and map. Use the learning rate adjustment strategy to fine-tune the network. Learning rate adjustment refers to using the Warmup method to warm up the network in the early stage of model training to prevent loss oscillation. After the loss curve is smoothed, the cosine decay strategy is used to smooth the learning rate and make the network jump out of the local optimal solution. The loss includes IOU loss, DFL loss and category loss. Label smoothing, as a regularization strategy, can reduce the network's reliance on label confidence, has good adaptability to missed or mislabeled data, and reduces the impact of problem labels on model accuracy.
[0078] The entire network is divided into a feature extraction network and three YOLO prediction branches. After freezing the feature trunk and neck network, two of the three predictions of the YOLO detection layer are frozen, and the remaining YOLO prediction layer is fine-tuned. When the model is trained to the set number of iterations, stop training and save the model and training parameters. The number of iterations is set to 300. The loss curve during training is as follows Figure 3 shown.
[0079] S4. Determine the confidence and bounding box in the prediction result, sort the bounding boxes based on the confidence, and select the bounding box with the highest confidence as the reference box.
[0080] The test set is input into the model optimized in step 6. After the test image is inferred by the model, a series of positive sample prediction boxes are obtained. The non-maximum suppression method (nms) is used for analysis, and the IOU threshold is set to 0.45.
[0081] First, the model outputs multiple prediction results including bounding boxes, categories, and confidence, where confidence is a comprehensive score of object confidence and category confidence. The predicted bounding boxes are sorted in descending order according to their confidence, ensuring that the boxes with the highest scores are processed first.
[0082] Then, the bounding box with the highest confidence is selected from the sorted list as the reference, and the intersection-over-union ratio of the reference box and all other boxes in the list is calculated (the model uses wise-iou). The wise-iou calculation method is as follows:
[0083] If the IoU of other boxes with the reference box exceeds the preset threshold iou_thres, these boxes are considered to overlap too much with the reference box and should be suppressed. After removing the suppressed boxes, repeatedly select the new box with the highest confidence and perform suppression until all boxes are calculated.
[0084] S5. Filter the bounding box through the reference box and output the recognition result.
[0085] The confidence of the predicted target for the five openness categories of first degree, second degree, third degree, fourth degree, and deformity is calculated through binary cross entropy loss, and the confidence is sorted. The category with the highest confidence is marked as the openness level of the recognition target, and the recognition result is output as follows Figure 4 shown.
[0086] The packaging technology of fresh cut flowers in China is corrugated paper packaging, with 15-20 petals in a bundle. The outer petals are affected by the packaging technology, and the degree of expansion in the packaging is unnatural. Refer to GB / T 41201-2021 for the openness classification standard of roses, which belong to the same family as roses, and combine it with the openness standard to make specifications, as shown in the following table.
[0087]
[0088] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms include, include or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0090] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for grading the degree of openness of fresh cut rose flowers, characterized in that: The following steps are involved: Obtain cut flower images and generate data sets after preprocessing; Annotate the data set and conduct visual analysis, and perform data expansion on deformed category images of the data set; Construct a neural network model for feature extraction and classification and train and optimize it, input the data set into the optimization model, and obtain the prediction result output by the optimization model. The optimization model uses Mobilenetv3 as a feature extraction module, optimizes the feature extraction, and introduces the SE channel attention mechanism and the EMA attention mechanism; Determining confidence levels and bounding boxes in the prediction results, and sorting the bounding boxes based on the confidence levels, and selecting the bounding box with the highest confidence level as a reference box; The bounding box is filtered by the reference box, and a recognition result is output.
2. The rose fresh cut flower opening degree classification method according to claim 1, wherein The neural network model includes: A feature extraction module, used to extract features of the data set and perform convolution, batch normalization and depth-separable convolution processing on the features; Neck module, including an upsampling part and a downsampling part, the upsampling part is used to obtain feature information of different scales, and the downsampling part is used to fuse the feature information and input the feature map into the corresponding detection head to predict the target; The prediction module includes three detection heads, each of which includes two branches, and the branches are used to calculate position loss and category loss.
3. The rose fresh cut flower opening degree classification method according to claim 2, wherein The position loss includes intelligent intersection-over-union loss and local feature loss. The intelligent intersection-over-union loss includes primary WIoU loss, secondary WIoU loss and tertiary WIoU loss. The primary WIoU loss introduces a normalized distance gain construction between the center point of the predicted box and the real box on the basis of the intersection-over-union loss. The secondary WIoU loss introduces an outlier construction on the basis of the primary WIoU loss. The tertiary WIoU loss introduces a hyperparameter construction on the basis of the secondary WIoU loss.
4. The rose fresh cut flower opening degree classification method according to claim 3, wherein The first-level WIoU loss L WIoUv1 , Secondary WIoU loss L WIoUv2 And the three-level WIoU loss L WIoUv3 It is expressed as: L WIoUv1 =R WIoU L IoU L WIoUv2 =(b) γ L WIoUv1 In the formula, R WIoU is the normalized distance gain between the predicted box and the center point of the real box, L IoU is the intersection-over-union loss, β is the outlier degree describing the quality of the anchor box, and σ and α are two hyperparameters for adjusting the β gain.
5. The rose fresh cut flower opening degree classification method according to claim 1, characterized in that, The filtering of the bounding box based on the reference box comprises: Calculate the intersection-over-union ratio of the reference frame and all other bounding boxes in the list; If the overlap index between the current bounding box and the reference box exceeds a preset overlap threshold, suppressing and removing the current bounding box; Repeatedly selecting the new reference box with the highest confidence and performing suppression until all the bounding boxes are calculated.
6. The rose fresh cut flower opening degree classification method according to claim 1, characterized in that: The output recognition result includes: Calculating the category confidence of the bounding box for five openness categories: first degree, second degree, third degree, fourth degree, and deformity; The category confidences are sorted and the category with the highest confidence is marked as the openness level of the identification target.
7. The rose fresh cut flower opening degree classification method according to claim 1, characterized in that: The labeling of the data set and visual analysis include: Annotating the images in the data set with openness categories and bounding boxes, wherein the bounding boxes are real boxes; The annotated images are visualized and analyzed to obtain deformed category images, where the deformed category images are category images in which the number of true frames is less than half of the average value of all category true frames.
8. The rose fresh cut flower opening degree classification method according to claim 1, characterized in that: The data expansion of the deformed category images of the data set includes: Obtaining the original image of the deformed image; Select the original image or cut out the real frame of the original image; The transformed image is adjusted by adding noise, random rotation, random flipping, and adding image distortion; The transformed images were added to the dataset.
9. The rose fresh cut flower opening degree classification method according to claim 1, characterized in that: Training and optimizing the neural network model includes: Inputting a training set into the neural network model; After training, cross validation is performed, and after calculating the accuracy index, a learning rate adjustment strategy is adopted and the network is fine-tuned; When the model is trained to the set number of iterations, the training is stopped and the model and training parameters are saved.
10. The rose fresh cut flower opening degree grading method according to claim 1, characterized in that: The EMA attention mechanism performs parallel pooling operations on feature maps in wide and high dimensions, expands the feature space through parallel 3×3 convolutions, and aggregates the three branches across space after processing.
Citation Information
Patent Citations
Tomato maturity detection method based on lightweight improved YOLO
CN114943697A
Flower openness detection and evaluation method
CN117094997A
General target detection method for adaptive attention guidance mechanism
WO2021139069A1
Cited By
Intelligent rose picking and detecting method
CN120877119A
Fresh cut flower grading method and system based on machine vision
CN121170313A
