A computer vision-based method and system for intelligent fruit grading

Abstract

The application discloses a fruit intelligent grading method based on computer vision. First, fruit color images are collected, and multi-dimensional quality features such as color maturity, texture luster, geometric shape and surface defects are extracted through a multi-branch parallel feature extraction network. Then, a defect fine discrimination network guided by maturity conditions is used to generate a defect discrimination convolution kernel according to maturity information, so as to eliminate the interference of maturity on defect recognition. Through a quality attribute adaptive weighted fusion network, the contribution weight of each quality attribute feature is dynamically learned, and a comprehensive quality feature vector is output. Finally, a cascaded multi-granularity grading network is used to realize hierarchical fine grading from coarse to fine, and the accurate quality grade of the fruit is output. Through multi-dimensional feature collaborative analysis and intelligent fusion, the fruit grading accuracy, robustness and fineness are significantly improved, and an efficient and reliable technical scheme is provided for fruit commercialization processing.

Description

Technical Field

[0001] This invention relates to the technical field of fruit grading, and in particular to a computer vision-based intelligent fruit grading method and system. Background Technology

[0002] As an important economic crop and consumer commodity, the quality grading of fruit directly affects its market value and consumer satisfaction. With the rapid development of modern agricultural industrialization and e-commerce logistics, the commercial processing of fruit places higher demands on grading efficiency and accuracy. Traditional manual grading methods rely on experienced sorting workers to judge the appearance, color, size, shape, and defects of fruit through visual observation and touch. This method has inherent drawbacks such as high labor intensity, low grading efficiency, inconsistent standards, and strong subjectivity, making it difficult to meet the needs of large-scale production and standardized management. Especially against the backdrop of continuously rising labor costs and labor shortages, the limitations of traditional manual grading models are becoming increasingly apparent, urgently requiring the introduction of intelligent technologies to automate and standardize fruit grading.

[0003] In recent years, with the rapid development of computer vision technology and deep learning algorithms, machine vision-based intelligent fruit grading methods have gradually become a research hotspot and application trend. Existing machine vision grading methods mainly acquire fruit images, extract visual features such as color, texture, and shape, and use classification algorithms to determine quality grades. However, existing methods still face many technical bottlenecks and performance limitations in practical applications.

[0004] First, existing methods generally employ single feature extraction strategies or simple feature stacking, failing to fully consider the multidimensionality and complexity of fruit quality evaluation. Fruit quality involves multiple aspects such as ripeness, color, texture, shape, size, and defects. These quality attributes differ significantly in physical characteristics and visual appearance, requiring targeted feature extraction methods. However, existing methods often use a uniform feature extraction network to process all quality attributes, resulting in limited expressive power for different attribute features and difficulty in accurately characterizing quality features across various dimensions, thus affecting the accuracy and reliability of grading.

[0005] Secondly, existing methods suffer from maturity interference in defect detection. Natural physiological phenomena on the fruit surface caused by changes in ripeness, such as natural spots and color transitions, are easily misidentified as defects, leading to a high false positive rate. This is because traditional defect detection algorithms typically rely on fixed color thresholds or texture features, failing to adaptively adjust defect discrimination criteria based on the fruit's ripeness. For example, green areas on unripe fruit and brown spots on overripe fruit can both be incorrectly identified as defects, affecting the accuracy of grading results.

[0006] Furthermore, existing methods lack adaptability and intelligence in the multi-feature fusion stage, typically employing simple feature splicing or fixed-weighting to integrate different quality attributes. However, the importance of different quality attributes in comprehensive quality evaluation dynamically changes with fruit type, ripening stage, and application scenario. For example, for gift fruits where appearance is the primary evaluation criterion, shape and color features should have higher weight; while for processed fruits where edibility is the primary evaluation criterion, ripeness and defect features are more crucial. Fixed feature fusion strategies cannot adapt to this dynamic change, limiting the flexibility and applicability of the grading model.

[0007] Finally, existing grading methods generally have a coarse granularity, mostly only achieving simple three- or four-level classifications such as excellent, good, and poor, which is insufficient to meet the demands of the high-end market for refined grading. The fruit market's requirements for quality grading are becoming increasingly refined, necessitating not only differentiation of basic grades but also more detailed sub-grades within each basic grade. For example, within superior fruit, further sub-grades such as extra-superior, superior+, superior, and superior- are needed to achieve differentiated pricing and precise market matching. Existing single-layer classification network architectures struggle to simultaneously ensure both overall accuracy and local precision in grading, performing poorly on fine-grained grading tasks.

[0008] In summary, existing intelligent fruit grading methods have significant shortcomings in areas such as accurate extraction of multi-dimensional features, precise defect identification under maturity conditions, adaptive fusion of quality attributes, and multi-granular fine grading. These shortcomings limit the practical application and promotional value of intelligent fruit grading technology. Therefore, there is an urgent need to develop a more intelligent, accurate, and flexible fruit grading method to overcome the limitations of existing technologies and meet the pressing needs of the modern fruit industry for intelligent grading. Summary of the Invention

[0009] In view of this, the present invention provides a computer vision-based intelligent fruit grading method and system. The purpose is to achieve accurate extraction of multi-dimensional quality features such as color, texture, shape and defects by constructing a multi-branch parallel feature extraction network, eliminate the interference of maturity on defect identification by a maturity condition-guided fine defect discrimination network, achieve intelligent integration of multi-dimensional features by a quality attribute adaptive weighted fusion network, and achieve hierarchical fine grading from coarse to fine by a cascaded multi-granularity grading network. This significantly improves the accuracy, robustness and precision of intelligent fruit grading, and provides an efficient and reliable technical solution for intelligent grading in the commercial processing of fruits.

[0010] To achieve the above objectives, the present invention provides a computer vision-based intelligent fruit grading method, comprising the following steps: S1: Acquire color images of the fruits to be graded, preprocess the color images, and input them into a multi-branch parallel feature extraction network. The multi-branch parallel feature extraction network includes a color branch, a texture branch, a morphology branch, and a defect branch. The color branch extracts the spatial distribution and maturity features of the fruit color and outputs a color maturity feature map. The texture branch extracts the surface texture and gloss features of the peel and outputs a texture and gloss feature map. The morphology branch extracts the geometric shape and size ratio features of the fruit outline and outputs a geometric shape feature map. The defect branch extracts the surface blemishes, spots, and mechanical damage area features and outputs an initial defect area feature map. S2: The color maturity feature map is used as a conditional signal input to the maturity-condition-guided fine defect discrimination network. In the maturity-condition-guided fine defect discrimination network, the dynamic convolution kernel generation module first adaptively generates defect discrimination convolution kernel weights for the current maturity stage based on the color maturity feature map. Then, the defect discrimination convolution kernel weights are used to perform layer-by-layer conditional convolution operations on the initial defect region feature map, and the accurate defect feature map after maturity correction is output. S3: The color maturity feature map, texture gloss feature map, geometric shape feature map and precise defect feature map are input into the quality attribute adaptive weighted fusion network. In the quality attribute adaptive weighted fusion network, the contribution weight of each quality attribute feature in the comprehensive quality evaluation is dynamically learned through the cross-attribute channel attention module. The four sets of feature maps are adaptively weighted and nonlinearly mapped to output a unified comprehensive quality feature vector. S4: Input the comprehensive quality feature vector into the cascaded multi-granularity grading network. The cascaded multi-granularity grading network includes a coarse grading module and a fine grading module. The coarse grading module divides the fruit into four basic grades: excellent, good, average, and unqualified. The fine grading module further subdivides each basic grade into sub-grades, and finally outputs the precise quality grade of the fruit, thus completing the intelligent grading of the fruit.

[0011] As a further improvement of the present invention: Optionally, step S1 further includes: S101: Perform background removal, noise reduction, and size normalization on the acquired color images of the fruits to be graded to obtain normalized color images. ,in The shape is 3 represents the red channel, green channel, and blue channel; S102: In the color branch, normalize the color image. Obtain an HSV color space image by converting from RGB color space to HSV color space. The color space distribution features are extracted using a multi-scale convolutional neural network, and then mapped to maturity quantification features through a fully connected layer, outputting a color maturity feature map. ,in The shape is , This represents the height dimension of the feature map. This represents the width dimension of the feature map. The number of channels in the color maturity feature map; S103: In the texture branch, construct a multi-directional Gabor filter bank, the multi-directional Gabor filter bank comprising... Different directions and angles and Gabor filters with different frequency parameters are used for normalized color images. Convolutional operations are performed to extract the texture response map, and then a texture encoding convolutional neural network is used to extract gloss features, outputting a texture gloss feature map. ,in The shape is , The number of channels in the texture gloss feature map; S104: In the morphological branch, the Canny edge detection algorithm is used to process the normalized color image. Edge detection is performed, and then a contour tracking algorithm is used to extract the outer contour coordinate sequence of the fruit. Geometric parameters, including the area of ​​the fruit, are calculated based on the outer contour coordinate sequence of the fruit. ,perimeter The length of the major axis of the smallest bounding rectangle and minor axis length The geometric parameters are mapped into one-dimensional morphological feature vectors through a fully connected network for morphological feature encoding, and then replicated and expanded in the spatial dimension to output a geometric morphological feature map. ,in The shape is , The number of channels representing the geometric morphology feature map; S105: In the defect branch, the normalized color image is processed by a defect detection convolutional neural network. Perform pixel-by-pixel classification to identify surface defects, spots, and mechanical damage areas, and output an initial defect area feature map. ,in The shape is , This represents the number of channels in the initial defect region feature map.

[0012] Optionally, step S2 further includes: S201: Color maturity feature map The input is fed into the dynamic convolutional kernel generation module, which first processes the color maturity feature map. Perform global average pooling to obtain the global maturity description vector. Then, a dynamic weight parameter matrix is ​​generated through a two-layer fully connected neural network. The calculation formula is: ; in, The weight matrix for the first layer of the dynamic convolution kernel generation module. This is the bias vector for the first layer of the dynamic convolution kernel generation module. It is the ReLU activation function. This is the weight matrix for the second layer of the dynamic convolution kernel generation module. The bias vector for the second layer of the dynamic convolution kernel generation module; S202: Using the dynamic weight parameter matrix The defect-discriminating convolutional kernel weights are generated through tensor reshaping operations to adapt to the current maturity stage. The calculation formula is: ; in, For tensor reshaping operations, This represents the number of output channels of a two-dimensional convolutional neural network. The number of input channels of a two-dimensional convolutional neural network and , The height dimension of the convolution kernel. The width dimension of the convolution kernel; S203: Use the generated defects to determine the convolutional kernel weights Feature map of the initial defect region Perform a two-dimensional conditional convolution operation to obtain an accurate defect feature map after maturity correction. The calculation formula is: ; in, Represents precise defect feature maps In the Each output channel, height position Width position eigenvalues ​​at that location Characteristic map of the initial defect region In the One input channel, height position Width position eigenvalues ​​at that location The weights of the defect-discriminating convolution kernels represent the weights of the convolution kernels. In the The output channel, the first Each input channel, convolution kernel height position Kernel width and position The weight value at that location.

[0013] This step achieves precise discrimination and accurate separation of defect features through a maturity-guided mechanism. Traditional defect detection methods typically employ fixed discrimination criteria, failing to adjust the discrimination strategy according to the fruit's ripening stage. This leads to misclassifying normal appearance changes during natural ripening as defects, or misclassifying true defects as normal ripening features. The dynamic convolutional kernel generation mechanism in this step adaptively generates defect-discriminating convolutional kernels based on the input color maturity feature map, ensuring that the defect discrimination process fully considers the current ripening state of the fruit.

[0014] Optionally, step S3 further includes: S301: Color maturity feature map Texture and gloss feature diagram Geometric morphological feature diagram and precise defect feature map Perform a stitching operation along the channel dimension to obtain a stitching quality feature map. Its shape is ,in , This represents the total number of channels in the splicing quality feature map; S302: Assemble the quality feature map The input is fed into the cross-attribute channel attention module to stitch the quality feature map. Perform global average pooling to obtain the channel description vector. The contribution weights of each quality attribute feature channel are learned through a two-layer fully connected neural network, and the channel attention weight vector is calculated. The calculation formula is: ; in, This is the weight matrix of the first fully connected layer of the cross-attribute channel attention module. This is the bias vector of the first fully connected layer of the cross-attribute channel attention module. This is the weight matrix of the second fully connected layer of the cross-attribute channel attention module. Here, represents the bias vector of the second fully connected layer of the cross-attribute channel attention module, and sigmoid represents the sigmoid activation function. For global average pooling; S303: Using Channel Attention Weight Vectors Feature diagram of splicing quality Channel-wise weighting is performed, followed by nonlinear mapping and dimensionality reduction through feature fusion convolutional layers and global average pooling layers to obtain a unified comprehensive quality feature vector. The calculation formula is: ; Where ⊙ represents element-wise multiplication, and Expand represents expanding the channel attention weight vector. Expanding in spatial dimension to Operations with the same shape The kernel weights of the feature fusion convolutional layer, The bias vector of the feature fusion convolutional layer. Convolution, or Flatten, is an operation that flattens a multidimensional tensor into a one-dimensional vector. The shape is , The feature dimension represents the feature vector of the overall quality.

[0015] This step utilizes an adaptive weighted fusion mechanism based on quality attributes to optimize and integrate multi-dimensional quality features, significantly improving the accuracy and robustness of the comprehensive quality evaluation. Traditional methods typically employ simple feature concatenation or fixed-weighting for multi-attribute feature fusion, failing to dynamically adjust the contribution of each attribute feature according to the quality characteristics of different fruit samples, resulting in redundant information and noise interference in the fused features. The cross-attribute channel attention mechanism in this step adaptively learns the importance weights of each quality attribute in the comprehensive quality evaluation based on the input multi-dimensional quality features, achieving differentiated weighted fusion of different quality attribute features.

[0016] Optionally, step S4 further includes: S401: Integrate the quality feature vector The input is fed into the coarse-leveling module, where the comprehensive quality feature vector is calculated through the coarse-leveling fully connected layer. The original score vectors belonging to the four basic levels of excellent, good, average, and unsatisfactory. Then, the Softmax function is used to convert it into the conditional probability distribution of each basic level. The base level with the highest probability value is selected as the coarse classification result. The calculation formula is: ; ; ; in, This is the weight matrix of the fully connected layer of the coarse-level module. This represents the bias vector of the fully connected layer in the coarse-level module. The shape [4] represents the scores of the four basic levels. Represents the conditional probability distribution The Middle The probability values ​​for each basic level. These correspond to four basic grades: excellent, good, average, and unqualified. Represents the original score vector The Middle The score values ​​for each basic level, Represents the original score vector The Middle The score values ​​for each basic level, The value range is from 1 to 4. This indicates that the base level index that maximizes the probability value should be returned. Exponential operations with the natural index as the base; S402: Based on the coarse grading results Generate the corresponding one-hot encoded vector , to integrate the quality feature vector With one-hot encoded vector Concatenate the features along the feature dimension to obtain the subdivided input feature vector. ; S403: Subdivide the input feature vector. The data is input to the sub-leveling module, which contains four sub-level classifiers corresponding to the four basic levels, based on the coarse-leveling results. Select the The sub-level classifier, through the first The fully connected layers of each sub-level classifier compute the sub-level input feature vectors. The original score vectors of each sub-level within the current base level Then, the Softmax function is used to convert it into the conditional probability distribution of each sub-level. The sub-level with the highest probability value is selected as the sub-level result. Original score vector The calculation formula is: ; in, For the first The weight matrix of the fully connected layer of the sub-level classifier corresponding to each base level. For the first The bias vectors of the fully connected layers of the sub-level classifiers corresponding to each base level; S404: Transform the coarse grading results Compared with subdivision results Combine and output the precise quality grade of the fruit. This enables intelligent grading of fruits.

[0017] This invention also discloses a computer vision-based intelligent fruit grading system, comprising: Feature extraction module: Acquires color images of fruits to be graded, preprocesses the color images and inputs them into a multi-branch parallel feature extraction network, which includes color branch, texture branch, morphology branch and defect branch; Defect Correction Module: The color maturity feature map is used as a conditional signal input to the maturity condition-guided defect fine discrimination network, and the output is an accurate defect feature map after maturity correction; Feature fusion module: The color maturity feature map, texture gloss feature map, geometric shape feature map and accurate defect feature map are input together into the quality attribute adaptive weighted fusion network, and the output is a unified comprehensive quality feature vector; Grading Module: The comprehensive quality feature vector is input into a cascaded multi-granularity grading network, which includes a coarse grading module and a fine grading module. Finally, the precise quality grade of the fruit is output, thus completing the intelligent grading of the fruit.

[0018] Compared with the prior art, the present invention has at least the following beneficial effects: This invention achieves accurate characterization and comprehensive extraction of multidimensional quality features of fruits through a multi-branch parallel feature extraction network, significantly improving the completeness and accuracy of feature representation. Unlike existing methods that use a single network to extract all features, this invention designs dedicated feature extraction branches for four different quality attributes: color, texture, morphology, and defects. Each branch employs a targeted algorithm and network structure. The color branch, through RGB-to-HSV color space conversion and a multi-scale convolutional neural network, can accurately extract the spatial distribution features of fruit color and quantitative features of ripeness. The texture branch, by constructing a Gabor filter bank containing multiple directions and frequencies, can finely capture the texture patterns and gloss features of the fruit peel surface. The morphology branch, through Canny edge detection and contour tracking algorithms, can accurately calculate the geometric parameters of the fruit and encode them into morphological features. The defect branch, through an encoder-decoder structured convolutional neural network, can effectively identify surface blemishes, spots, and mechanically damaged areas.

[0019] This invention innovatively solves the maturity interference problem in defect detection through a maturity-guided fine-grained defect discrimination network, significantly reducing the false positive rate and improving the accuracy of defect identification. Existing methods commonly misclassify natural color spots caused by maturity as defects, leading to a high false positive rate. This invention uses a dynamic convolutional kernel generation module to adaptively generate defect discrimination convolutional kernel weights based on the color maturity feature map, enabling the defect detection network to dynamically adjust its feature extraction strategy and discrimination criteria according to the fruit's maturity state. Specifically, for immature fruit, the generated convolutional kernel weights reduce sensitivity to green areas, avoiding misclassifying normal immature green areas as defects; for overripe fruit, the generated convolutional kernel weights improve the ability to identify real mechanical damage and disease spots, while tolerating normal ripening browning.

[0020] This invention achieves intelligent fusion evaluation and refined hierarchical grading of fruit quality through the collaborative design of an adaptive weighted fusion network for quality attributes and a cascaded multi-granularity grading network, significantly improving the flexibility and practicality of the grading system. In the feature fusion stage, this invention employs a cross-attribute channel attention module to dynamically learn the contribution weights of each quality attribute feature in the comprehensive quality evaluation. The feature fusion strategy is adaptively adjusted according to different fruit types, application scenarios, and quality focuses, avoiding the rigidity of fixed-weight fusion and ensuring that the fused comprehensive quality feature vector more accurately reflects the true quality status of the fruit. In the grading stage, this invention uses a cascaded multi-granularity grading network. First, a coarse grading module divides the fruit into four basic grades: excellent, good, average, and unqualified. Then, a sub-grading module further subdivides each basic grade into sub-grades, achieving a hierarchical and refined grading from macro to micro levels. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a computer vision-based intelligent fruit grading method according to an embodiment of the present invention. Figure 2 A schematic diagram comparing RGB images and defect feature maps of mango surface defects: (a) RGB image, (b) schematic diagram of accurate defect feature map after maturity correction. Detailed Implementation

[0022] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0023] Example 1: A computer vision-based intelligent fruit grading method, such as... Figure 1 As shown, it includes the following steps: S1: Acquire color images of the fruit to be graded. After preprocessing the color images, input them into a multi-branch parallel feature extraction network. The multi-branch parallel feature extraction network includes a color branch, a texture branch, a morphology branch, and a defect branch. The color branch extracts the spatial distribution and maturity features of the fruit's color and outputs a color maturity feature map. The texture branch extracts the surface texture and gloss features of the peel and outputs a texture and gloss feature map. The morphology branch extracts the geometric shape and size ratio features of the fruit's outline and outputs a geometric shape feature map. The defect branch extracts surface blemishes, spots, and mechanical damage area features and outputs an initial defect area feature map, including: S101: For the collected color images of the fruits to be graded, such as... Figure 2 As shown in (a), background removal, noise reduction, and size normalization are performed to obtain a normalized color image. ,in The shape is 3 represents the red channel, green channel, and blue channel; S102: In the color branch, normalize the color image. Obtain an HSV color space image by converting from RGB color space to HSV color space. The color space distribution features are extracted using a multi-scale convolutional neural network, and then mapped to maturity quantification features through a fully connected layer, outputting a color maturity feature map. ,in The shape is , This represents the height dimension of the feature map. This represents the width dimension of the feature map. This represents the number of channels in the color maturity feature map. In this embodiment, the multi-scale convolutional neural network contains three parallel convolutional branches, using convolutional kernels of three different sizes: 3×3, 5×5, and 7×7. Each branch has 64 kernels, a stride of 1, and uses the same padding method to maintain the feature map size. The output feature maps of the three convolutional branches are concatenated along the channel dimension to obtain a 192-channel multi-scale color feature map. Then, a 1×1 convolutional layer is used for channel compression, and the output channel number is... Color maturity characteristic map In this embodiment, the spatial size of the feature map Pixels Pixels are downsampled from an input size of 224×224 to a feature map size of 28×28 by inserting a max pooling layer in a multi-scale convolutional neural network; S103: In the texture branch, construct a multi-directional Gabor filter bank, the multi-directional Gabor filter bank comprising... Different directions and angles and Gabor filters with different frequency parameters are used for normalized color images. Convolutional operations are performed to extract the texture response map, and then a texture encoding convolutional neural network is used to extract gloss features, outputting a texture gloss feature map. ,in The shape is , This represents the number of channels in the texture gloss feature map. In this embodiment, the texture encoding convolutional neural network includes three convolutional layers. The first convolutional layer has 32 input channels, a kernel size of 5×5, 64 kernels, and a stride of 2. The second convolutional layer has 64 input channels, a kernel size of 3×3, 128 kernels, and a stride of 2. The third convolutional layer has 128 input channels, a kernel size of 3×3, and a stride of 2. With a stride of 2, the feature map size is downsampled from 224×224 to 28×28 through three convolution operations with a stride of 2, outputting a texture gloss feature map. The shape is Each convolutional layer is followed by a BatchNormalization layer and a ReLU activation function. S104: In the morphological branch, the Canny edge detection algorithm is used to process the normalized color image. Edge detection is performed, and then a contour tracking algorithm is used to extract the outer contour coordinate sequence of the fruit. Geometric parameters, including the area of ​​the fruit, are calculated based on the outer contour coordinate sequence of the fruit. ,perimeter The length of the major axis of the smallest bounding rectangle and minor axis length The geometric parameters are mapped into one-dimensional morphological feature vectors through a fully connected network for morphological feature encoding, and then replicated and expanded in the spatial dimension to output a geometric morphological feature map. ,in The shape is , This represents the number of channels in the geometric morphological feature map. In this embodiment, the morphological feature encoding fully connected network contains two fully connected layers. The first fully connected layer has an input dimension of 4, an output dimension of 64, and uses ReLU as the activation function. The second fully connected layer has an input dimension of 64 and an output dimension of... The activation function is ReLU, resulting in a one-dimensional morphological feature vector. This one-dimensional feature vector is then copied and expanded spatially through a tensor copy operation to obtain a shape of... Geometric morphological feature diagram ; S105: In the defect branch, the normalized color image is processed by a defect detection convolutional neural network. Perform pixel-by-pixel classification to identify surface defects, spots, and mechanical damage areas, and output an initial defect area feature map. ,in The shape is , This represents the number of channels in the initial defect region feature map. In this embodiment, the defect detection convolutional neural network adopts an encoder-decoder structure. The encoder part contains four convolutional blocks, each consisting of two 3×3 convolutional layers, one batchNormalization layer, one ReLU activation function, and one 2×2 max pooling layer. The number of output channels for each convolutional block of the encoder is 64, 128, 256, and 512, respectively. The input image size is downsampled from 224×224 to 14×14 through four max pooling operations. The decoder part contains four upsampling blocks, each consisting of a 2×2 transposed convolutional layer, two 3×3 convolutional layers, one batchNormalization layer, and one ReLU activation function. The number of output channels for each upsampling block of the decoder is 256, 128, and 64, respectively. The feature map size is upsampled from 14×14 to 224×224 through four transposed convolution operations. To maintain the feature map size consistent with other branches, an average pooling layer with a stride of 8 is added at the end of the decoder to downsample the feature map size from 224×224 to 28×28, outputting the initial defect region feature map. The shape is Shallow features are passed between the encoder and decoder via skip connections, and the feature maps of each layer of the encoder are concatenated with the feature maps of the corresponding layers of the decoder along the channel dimension. S2: The color maturity feature map is input as a conditional signal to the maturity-condition-guided fine-grained defect discrimination network. In the maturity-condition-guided fine-grained defect discrimination network, the dynamic convolutional kernel generation module first adaptively generates defect discrimination convolutional kernel weights for the current maturity stage based on the color maturity feature map. Then, the defect discrimination convolutional kernel weights are used to perform layer-by-layer conditional convolution operations on the initial defect region feature map, outputting a precise defect feature map after maturity correction, including: S201: Color maturity feature map The input is fed into the dynamic convolutional kernel generation module, which first processes the color maturity feature map. Perform global average pooling to obtain the global maturity description vector. Then, a dynamic weight parameter matrix is ​​generated through a two-layer fully connected neural network. The calculation formula is: ; in, The weight matrix for the first layer of the dynamic convolution kernel generation module. This is the bias vector for the first layer of the dynamic convolution kernel generation module. It is the ReLU activation function. This is the weight matrix for the second layer of the dynamic convolution kernel generation module. The bias vector for the second layer of the dynamic convolution kernel generation module; S202: Using the dynamic weight parameter matrix The defect-discriminating convolutional kernel weights are generated through tensor reshaping operations to adapt to the current maturity stage. The calculation formula is: ; in, For tensor reshaping operations, This represents the number of output channels of a two-dimensional convolutional neural network. The number of input channels of a two-dimensional convolutional neural network and , The height dimension of the convolution kernel. The width dimension of the convolution kernel; S203: Use the generated defects to determine the convolutional kernel weights Feature map of the initial defect region Perform a two-dimensional conditional convolution operation to obtain an accurate defect feature map after maturity correction. ,like Figure 2 As shown in (b), the calculation formula is: ; in, Represents precise defect feature maps In the Each output channel, height position Width position eigenvalues ​​at that location Characteristic map of the initial defect region In the One input channel, height position Width position eigenvalues ​​at that location The weights of the defect-discriminating convolution kernels represent the weights of the convolution kernels. In the The output channel, the first Each input channel, convolution kernel height position Kernel width and position The weight value at that location.

[0024] S3: The color maturity feature map, texture gloss feature map, geometric morphology feature map, and precise defect feature map are input into the quality attribute adaptive weighted fusion network. In the quality attribute adaptive weighted fusion network, the contribution weight of each quality attribute feature in the comprehensive quality evaluation is dynamically learned through a cross-attribute channel attention module. The four sets of feature maps are adaptively weighted and nonlinearly mapped to output a unified comprehensive quality feature vector, including: S301: Color maturity feature map Texture and gloss feature diagram Geometric morphological feature diagram and precise defect feature map Perform a stitching operation along the channel dimension to obtain a stitching quality feature map. Its shape is ,in , This represents the total number of channels in the splicing quality feature map; S302: Assemble the quality feature map The input is fed into the cross-attribute channel attention module to stitch the quality feature map. Perform global average pooling to obtain the channel description vector. The contribution weights of each quality attribute feature channel are learned through a two-layer fully connected neural network, and the channel attention weight vector is calculated. The calculation formula is: ; in, This is the weight matrix of the first fully connected layer of the cross-attribute channel attention module. This is the bias vector of the first fully connected layer of the cross-attribute channel attention module. This is the weight matrix of the second fully connected layer of the cross-attribute channel attention module. Here, represents the bias vector of the second fully connected layer of the cross-attribute channel attention module, and sigmoid represents the sigmoid activation function. For global average pooling; S303: Using Channel Attention Weight Vectors Feature diagram of splicing quality Channel-wise weighting is performed, followed by nonlinear mapping and dimensionality reduction through feature fusion convolutional layers and global average pooling layers to obtain a unified comprehensive quality feature vector. The calculation formula is: ; Where ⊙ represents element-wise multiplication, and Expand represents expanding the channel attention weight vector. Expanding in spatial dimension to Operations with the same shape The kernel weights of the feature fusion convolutional layer, The bias vector of the feature fusion convolutional layer. Convolution, or Flatten, is an operation that flattens a multidimensional tensor into a one-dimensional vector. The shape is , The feature dimension represents the feature vector of the overall quality.

[0025] S4: The comprehensive quality feature vector is input into a cascaded multi-granularity grading network. This network includes a coarse grading module and a fine grading module. The coarse grading module divides the fruit into four basic grades: excellent, good, average, and unqualified. The fine grading module then further subdivides each basic grade into sub-grades, ultimately outputting the precise quality grade of the fruit, thus completing intelligent fruit grading. This includes: S401: Integrate the quality feature vector The input is fed into the coarse-leveling module, where the comprehensive quality feature vector is calculated through the coarse-leveling fully connected layer. The original score vectors belonging to the four basic levels of excellent, good, average, and unsatisfactory. Then, the Softmax function is used to convert it into the conditional probability distribution of each basic level. The base level with the highest probability value is selected as the coarse classification result. The calculation formula is: ; ; ; in, This is the weight matrix of the fully connected layer of the coarse-level module. This represents the bias vector of the fully connected layer in the coarse-level module. The shape [4] represents the scores of the four basic levels. Represents the conditional probability distribution The Middle The probability values ​​for each basic level. These correspond to four basic grades: excellent, good, average, and unqualified. Represents the original score vector The Middle The score values ​​for each basic level, Represents the original score vector The Middle The score values ​​for each basic level, The value range is from 1 to 4. This indicates that the base level index that maximizes the probability value should be returned. This is an exponential operation with the natural exponent as the base; in this embodiment, the weight matrix of the fully connected layer of the coarse-leveling module... The shape is Bias vector The shape is , 256-dimensional comprehensive quality feature vector Mapped to a 4-dimensional original score vector Each dimension corresponds to a basic level of classification score; S402: Based on the coarse grading results Generate the corresponding one-hot encoded vector , to integrate the quality feature vector With one-hot encoded vector Concatenate the features along the feature dimension to obtain the subdivided input feature vector. ; S403: Subdivide the input feature vector. The data is input to the sub-leveling module, which contains four sub-level classifiers corresponding to the four basic levels, based on the coarse-leveling results. Select the The sub-level classifier, through the first The fully connected layers of each sub-level classifier compute the sub-level input feature vectors. The original score vectors of each sub-level within the current base level Then, the Softmax function is used to convert it into the conditional probability distribution of each sub-level. The sub-level with the highest probability value is selected as the sub-level result. Original score vector The calculation formula is: ; in, For the first The weight matrix of the fully connected layer of the sub-level classifier corresponding to each base level. For the first The bias vectors of the fully connected layers of the sub-level classifiers corresponding to the basic levels; in this embodiment, the sub-level module contains four independent sub-level classifiers, corresponding to the four basic levels of excellent, good, average, and unqualified, respectively; each sub-level classifier is responsible for subdividing the basic level within its corresponding basic level; in this embodiment, the excellent basic level is set to contain 3 sub-levels of excellent+, excellent, and excellent-, the good basic level is set to contain 3 sub-levels of good+, good, and good-, the average basic level is set to contain 2 sub-levels of average and poor, and the unqualified basic level is not subdivided, containing only 1 sub-level, unqualified; therefore, the output dimensions of the four sub-level classifiers are 3, 3, 2, and 1, respectively; based on the coarse grading results Select the corresponding sub-level classifier; S404: Transform the coarse grading results Compared with subdivision results Combine and output the precise quality grade of the fruit. This enables intelligent grading of fruits.

[0026] In this embodiment, the entire fruit intelligent grading network is jointly trained end-to-end. The loss function is defined as the weighted sum of the coarse grading loss and the fine grading loss, specifically: ,in For coarse classification loss, For detailed loss analysis, and These are the loss weighting coefficients; coarse-grained loss. and subdivision loss Both use the cross-entropy loss function; the loss weight coefficients are set to... , This assigns higher weights to the subdivision loss, prompting the model to pay more attention to the accuracy of the fine-grained classification.

[0027] In this embodiment, the optimizer used is the Adam optimizer. The hyperparameters of the Adam optimizer are set as follows: initial learning rate of 0.001, first-order moment decay rate of 0.9, second-order moment decay rate of 0.999, and numerical stability constant of . The total number of training cycles is 100. Training is terminated early if the loss function value does not decrease for 10 consecutive cycles.

[0028] Example 2: This invention also discloses a computer vision-based intelligent fruit grading system, comprising the following five modules: Feature extraction module: Acquires color images of fruits to be graded, preprocesses the color images and inputs them into a multi-branch parallel feature extraction network, which includes color branch, texture branch, morphology branch and defect branch; Defect Correction Module: The color maturity feature map is used as a conditional signal input to the maturity condition-guided defect fine discrimination network, and the output is an accurate defect feature map after maturity correction; Feature fusion module: The color maturity feature map, texture gloss feature map, geometric shape feature map and accurate defect feature map are input together into the quality attribute adaptive weighted fusion network, and the output is a unified comprehensive quality feature vector; Grading Module: The comprehensive quality feature vector is input into a cascaded multi-granularity grading network, which includes a coarse grading module and a fine grading module. Finally, the precise quality grade of the fruit is output, thus completing the intelligent grading of the fruit.

[0029] It should be noted that the sequence numbers of the above embodiments of the present invention are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0030] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0031] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A computer vision-based intelligent fruit grading method, characterized in that, Includes the following steps: S1: Acquire color images of the fruits to be graded, preprocess the color images, and input them into a multi-branch parallel feature extraction network. The multi-branch parallel feature extraction network includes a color branch, a texture branch, a morphology branch, and a defect branch. The color branch extracts the spatial distribution and maturity features of the fruit color and outputs a color maturity feature map. The texture branch extracts the surface texture and gloss features of the peel and outputs a texture and gloss feature map. The morphology branch extracts the geometric shape and size ratio features of the fruit outline and outputs a geometric shape feature map. The defect branch extracts the surface blemishes, spots, and mechanical damage area features and outputs an initial defect area feature map. S2: The color maturity feature map is used as a conditional signal input to the maturity-condition-guided fine defect discrimination network. In the maturity-condition-guided fine defect discrimination network, the dynamic convolution kernel generation module first adaptively generates defect discrimination convolution kernel weights for the current maturity stage based on the color maturity feature map. Then, the defect discrimination convolution kernel weights are used to perform layer-by-layer conditional convolution operations on the initial defect region feature map, and the accurate defect feature map after maturity correction is output. S3: The color maturity feature map, texture gloss feature map, geometric shape feature map and precise defect feature map are input into the quality attribute adaptive weighted fusion network. In the quality attribute adaptive weighted fusion network, the contribution weight of each quality attribute feature in the comprehensive quality evaluation is dynamically learned through the cross-attribute channel attention module. The four sets of feature maps are adaptively weighted and nonlinearly mapped to output a unified comprehensive quality feature vector. S4: Input the comprehensive quality feature vector into the cascaded multi-granularity grading network. The cascaded multi-granularity grading network includes a coarse grading module and a fine grading module. The coarse grading module divides the fruit into four basic grades: excellent, good, average, and unqualified. The fine grading module further subdivides each basic grade into sub-grades, and finally outputs the precise quality grade of the fruit, thus completing the intelligent grading of the fruit.

2. The intelligent fruit grading method based on computer vision according to claim 1, characterized in that, Step S1 includes: S101: Perform background removal, noise reduction, and size normalization on the acquired color images of the fruits to be graded to obtain normalized color images. ,in The shape is 3 represents the red channel, green channel, and blue channel; S102: In the color branch, normalize the color image. Obtain an HSV color space image by converting from RGB color space to HSV color space. The color space distribution features are extracted using a multi-scale convolutional neural network, and then mapped to maturity quantification features through a fully connected layer, outputting a color maturity feature map. ,in The shape is , This represents the height dimension of the feature map. This represents the width dimension of the feature map. The number of channels in the color maturity feature map; S103: In the texture branch, construct a multi-directional Gabor filter bank, the multi-directional Gabor filter bank comprising... Different directions and angles and Gabor filters with different frequency parameters are used for normalized color images. Convolutional operations are performed to extract the texture response map, and then a texture encoding convolutional neural network is used to extract gloss features, outputting a texture gloss feature map. ,in The shape is , The number of channels in the texture gloss feature map; S104: In the morphological branch, the Canny edge detection algorithm is used to process the normalized color image. Edge detection is performed, and then a contour tracking algorithm is used to extract the outer contour coordinate sequence of the fruit. Geometric parameters, including the area of ​​the fruit, are calculated based on the outer contour coordinate sequence of the fruit. ,perimeter The length of the major axis of the smallest bounding rectangle and minor axis length The geometric parameters are mapped into one-dimensional morphological feature vectors through a fully connected network for morphological feature encoding, and then replicated and expanded in the spatial dimension to output a geometric morphological feature map. ,in The shape is , The number of channels representing the geometric morphology feature map; S105: In the defect branch, the normalized color image is processed by a defect detection convolutional neural network. Perform pixel-by-pixel classification to identify surface defects, spots, and mechanical damage areas, and output an initial defect area feature map. ,in The shape is , This represents the number of channels in the initial defect region feature map.

3. The intelligent fruit grading method based on computer vision according to claim 2, characterized in that, Step S2 includes: S201: Color maturity feature map The input is fed into the dynamic convolutional kernel generation module, which first processes the color maturity feature map. Perform global average pooling to obtain the global maturity description vector. Then, a dynamic weight parameter matrix is ​​generated through a two-layer fully connected neural network. The calculation formula is: ； in, The weight matrix for the first layer of the dynamic convolution kernel generation module. This is the bias vector for the first layer of the dynamic convolution kernel generation module. It is the ReLU activation function. This is the weight matrix for the second layer of the dynamic convolution kernel generation module. The bias vector for the second layer of the dynamic convolution kernel generation module; S202: Using the dynamic weight parameter matrix The defect-discriminating convolutional kernel weights are generated through tensor reshaping operations to adapt to the current maturity stage. The calculation formula is: ； in, For tensor reshaping operations, This represents the number of output channels of a two-dimensional convolutional neural network. The number of input channels of a two-dimensional convolutional neural network and , The height dimension of the convolution kernel. The width dimension of the convolution kernel; S203: Use the generated defects to determine the convolutional kernel weights Feature map of the initial defect region Perform a two-dimensional conditional convolution operation to obtain an accurate defect feature map after maturity correction. The calculation formula is: ； in, Represents precise defect feature maps In the Each output channel, height position Width position eigenvalues ​​at that location Characteristic map of the initial defect region In the One input channel, height position Width position eigenvalues ​​at that location The weights of the defect-discriminating convolution kernels represent the weights of the convolution kernels. In the The output channel, the first Each input channel, convolution kernel height position Kernel width and position The weight value at that location.

4. The intelligent fruit grading method based on computer vision according to claim 3, characterized in that, Step S3 includes: S301: Color maturity feature map Texture and gloss feature diagram Geometric morphological feature diagram and precise defect feature map Perform a stitching operation along the channel dimension to obtain a stitching quality feature map. Its shape is ,in , This represents the total number of channels in the splicing quality feature map; S302: Assemble the quality feature map The input is fed into the cross-attribute channel attention module to stitch the quality feature map. Perform global average pooling to obtain the channel description vector. The contribution weights of each quality attribute feature channel are learned through a two-layer fully connected neural network, and the channel attention weight vector is calculated. The calculation formula is: ； in, This is the weight matrix of the first fully connected layer of the cross-attribute channel attention module. This is the bias vector of the first fully connected layer of the cross-attribute channel attention module. This is the weight matrix of the second fully connected layer of the cross-attribute channel attention module. Here, represents the bias vector of the second fully connected layer of the cross-attribute channel attention module, and sigmoid represents the sigmoid activation function. For global average pooling; S303: Using Channel Attention Weight Vectors Feature diagram of splicing quality Channel-wise weighting is performed, followed by nonlinear mapping and dimensionality reduction through feature fusion convolutional layers and global average pooling layers to obtain a unified comprehensive quality feature vector. The calculation formula is: ； Where ⊙ represents element-wise multiplication, and Expand represents expanding the channel attention weight vector. Expanding in spatial dimension to Operations with the same shape The kernel weights of the feature fusion convolutional layer, The bias vector of the feature fusion convolutional layer. Convolution, or Flatten, is an operation that flattens a multidimensional tensor into a one-dimensional vector. The shape is , The feature dimension represents the feature vector of the overall quality.

5. The intelligent fruit grading method based on computer vision according to claim 4, characterized in that, Step S4 includes: S401: Integrate the quality feature vector The input is fed into the coarse-leveling module, where the comprehensive quality feature vector is calculated through the coarse-leveling fully connected layer. The original score vectors belonging to the four basic levels of excellent, good, average, and unsatisfactory. Then, the Softmax function is used to convert it into the conditional probability distribution of each basic level. The base level with the highest probability value is selected as the coarse classification result. The calculation formula is: ； ； ； in, This is the weight matrix of the fully connected layer of the coarse-level module. This represents the bias vector of the fully connected layer in the coarse-level module. The shape [4] represents the scores of the four basic levels. Represents the conditional probability distribution The Middle The probability values ​​for each basic level. These correspond to four basic grades: excellent, good, average, and unqualified. Represents the original score vector The Middle The score values ​​for each basic level, Represents the original score vector The Middle The score values ​​for each basic level, The value range is from 1 to 4. This indicates that the base level index that maximizes the probability value should be returned. Exponential operations with the natural index as the base; S402: Based on the coarse grading results Generate the corresponding one-hot encoded vector , to integrate the quality feature vector With one-hot encoded vector Concatenate the features along the feature dimension to obtain the subdivided input feature vector. ; S403: Subdivide the input feature vector. The data is input to the sub-leveling module, which contains four sub-level classifiers corresponding to the four basic levels, based on the coarse-leveling results. Select the The sub-level classifier, through the first The fully connected layers of each sub-level classifier compute the sub-level input feature vectors. The original score vectors of each sub-level within the current base level Then, the Softmax function is used to convert it into the conditional probability distribution of each sub-level. The sub-level with the highest probability value is selected as the sub-level result. Original score vector The calculation formula is: ； in, For the first The weight matrix of the fully connected layer of the sub-level classifier corresponding to each base level. For the first The bias vectors of the fully connected layers of the sub-level classifiers corresponding to each base level; S404: Transform the coarse grading results Compared with subdivision results Combine and output the precise quality grade of the fruit. This enables intelligent grading of fruits.

6. A computer vision-based intelligent fruit grading system, characterized in that, include: Feature extraction module: Acquires color images of fruits to be graded, preprocesses the color images and inputs them into a multi-branch parallel feature extraction network, which includes color branch, texture branch, morphology branch and defect branch; Defect Correction Module: The color maturity feature map is used as a conditional signal input to the maturity condition-guided defect fine discrimination network, and the output is an accurate defect feature map after maturity correction; Feature fusion module: The color maturity feature map, texture gloss feature map, geometric shape feature map and accurate defect feature map are input together into the quality attribute adaptive weighted fusion network, and the output is a unified comprehensive quality feature vector; Grading Module: The comprehensive quality feature vector is input into the cascaded multi-granularity grading network, which includes a coarse grading module and a fine grading module, and finally outputs the precise quality grade of the fruit, thus completing the intelligent grading of the fruit. To achieve the intelligent fruit grading method based on computer vision as described in any one of claims 1-5.

Images