Blueberry maturity detection method and system based on computer vision

Abstract

The invention discloses a blueberry maturity detection method and system based on computer vision, and relates to the technical field of fruit quality detection. Comprising the following steps: acquiring a blueberry image in a 400-1000nm wave band through multiple spectrums, and carrying out denoising, distortion correction and region-of-interest segmentation to obtain a single blueberry independent image; color features are extracted based on principal component analysis of 400-700 nm wavebands, fruit powder features are comprehensively characterized by constructing fruit powder feature evaluation coefficients, and pedicel features are comprehensively characterized by constructing pedicel feature evaluation coefficients and shape features; outputting four levels of immaturity, primary maturity, medium maturity and maturity for the features through a maturity judgment model constructed by a dual-channel convolutional neural network model fusing an attention mechanism, and meanwhile, realizing model increment updating by combining a block chain and federated learning; according to the method, through multi-dimensional feature fusion and targeted model design, the accuracy and robustness of blueberry maturity detection are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of fruit quality detection, and particularly relates to a blueberry maturity detection method and system based on computer vision. BACKGROUND

[0002] Blueberry maturity detection is a key link to determine its harvesting time, shelf life and commodity value.

[0003] In the prior art, the traditional detection method relies on manual visual inspection, which has strong subjectivity, low efficiency and large error, and cannot meet the needs of large-scale production. At the same time, although the detection technology based on computer vision has realized the detection of blueberry maturity to some extent, it still has the following problems: It relies too much on single features such as color or shape, and ignores key indicators closely related to maturity such as fruit powder and fruit stem. For example, fruit powder is a protective wax layer on the surface of blueberries, and its coverage and thickness change dynamically with maturity. Existing methods often do not quantify it. The green fading, browning and elasticity change of the fruit stem are important indicators of maturity, but they are often simplified as color features. The general classification model is not optimized for the heterogeneity of blueberry features, and the feature fusion efficiency is low, which limits the detection accuracy. Agricultural data has privacy and dispersion, and traditional centralized training cannot integrate multi-source data. The model has weak generalization ability and cannot adapt to the detection needs of blueberries of different varieties and origins.

[0004] Therefore, there is an urgent need for a blueberry maturity detection method that can integrate multi-dimensional features, adapt to heterogeneous data and be dynamically optimized to improve the accuracy, efficiency and practicality of detection. SUMMARY

[0005] The present application aims to at least solve one of the technical problems in the prior art. To this end, the present application proposes a blueberry maturity detection method and system based on computer vision to solve the following technical problems: Relying too much on single features such as color or shape and ignoring key indicators closely related to maturity such as fruit powder and fruit stem. The extracted color features, fruit powder feature evaluation coefficients, fruit stem feature evaluation coefficients and shape feature heterogeneous data are difficult to efficiently fuse. Moreover, the classification model is not optimized for the heterogeneity of multiple blueberry features, and the feature fusion efficiency is low, which limits the detection accuracy. Agricultural data has privacy and dispersion, and traditional centralized training cannot integrate multi-source data. The model has weak generalization ability and cannot adapt to the detection needs of blueberries of different varieties and origins.

[0006] To solve the above problems, the first aspect of the present application provides a blueberry maturity detection method based on computer vision, comprising the following steps: S1: acquiring image information of blueberry samples from different angles by a multispectral camera, and the wave band covers 400-1000 nm; S2: denoising, distortion correction and region of interest segmentation are performed on the acquired images to obtain independent images of individual blueberries; S3: based on a preset feature extraction model, color features, fruit powder features, stem features and shape features are extracted from the independent images of individual blueberries, the color features are divided into 60 channels based on 400-700 nm wave band at 10 nm intervals, the first three principal components are retained by principal component analysis to form a color feature matrix, the fruit powder features are comprehensively represented by constructing a fruit powder feature evaluation coefficient, the stem features are comprehensively represented by constructing a stem feature evaluation coefficient, and the shape features are calculated to form a 3-dimensional shape feature vector by calculating its diameter, curvature and regularity; S4: the extracted features are input into a trained maturity determination model to output the maturity grade of the blueberries, wherein the fruit powder and stem feature evaluation coefficients are mapped into vectors consistent with the number of color feature channels through a feature embedding layer, and are input into the model together with other features.

[0007] Preferably, the step S2 comprises the following steps: Gaussian filtering combined with non-local mean denoising algorithm is used to remove noise, and distortion correction is performed based on the camera intrinsic matrix; The region of interest is segmented by fusing deep learning semantic segmentation and adaptive threshold segmentation, specifically: first, a pre-trained U-Net model is used to preliminarily locate the blueberry region, then the threshold parameter is dynamically adjusted based on the average reflectivity of the 650 nm wave band, and the Otsu algorithm is used to segment the independent images of individual blueberries.

[0008] Preferably, in the step S3, the fruit powder feature comprises the following steps: The improved U-Net++ network is used to segment the fruit powder region, specifically: the encoder introduces the inverted residual structure of MoblieNetV3, and the spectral attention module is added to the decoder; The segmentation results of the two are fused, and the white fruit powder region and the fruit entity region are segmented by the reflectivity feature of the 850 nm near-infrared wave band and the Otsu threshold segmentation result; The area ratio of the white fruit powder region on the surface of the blueberry is calculated as the blueberry white fruit powder coverage; The box dimension is calculated based on fractal geometry as the surface roughness of the blueberry; The uniformity of the distribution of the white fruit powder region on the surface of the blueberry is calculated, specifically: ; Wherein, U represents the uniformity, and sigmaA μ represents the local variance within the sliding window. A This represents the local mean within a sliding window, which is 5x5 pixels in size. The disparity map is generated by combining a stereo matching algorithm, and the 3D point cloud data is reconstructed by triangulation to measure the thickness of the fruit powder layer. Specifically: ; Where h(x,y) represents the thickness of the fruit powder layer at image coordinates (x,y), (x,y) represents the pixel coordinates in the image, and Z... surface Z represents the depth value of the fruit powder surface in three-dimensional coordinates. baseline This represents the depth value of the blueberry fruit's surface in three-dimensional coordinates.

[0009] Preferably, the construction of the fruit powder characteristic evaluation coefficient includes the following steps: The white bloom coverage, surface roughness, uniformity, and bloom layer thickness of blueberries were standardized respectively. The standardized feature parameters were then weighted and summed to construct a bloom characteristic evaluation coefficient, specifically: ; Where K represents the bloom characteristic evaluation coefficient, β1, β2, β3 and β4 represent the dynamic weights of blueberry white bloom coverage, standardized blueberry surface roughness, uniformity and average bloom layer thickness, respectively, and C represents blueberry white bloom coverage. U represents the standardized surface roughness of blueberries, and U represents uniformity. This indicates the average thickness of the bloom layer.

[0010] Preferably, in step S3, the stem feature includes the following steps: Heatmap of the result area generated by training the UNet++ network Where P represents the probability heatmap of the fruit stem region, H represents the height of the heatmap, and W represents the width of the heatmap. The improved U-Net image segmentation algorithm is used to determine the region where the fruit stem is located. The percentage of pixels with H values ​​in the HSV color space between 40° and 80° in the fruit stem region is calculated to obtain its green occupancy, and the dry crack density factor and browning index are extracted. The displacement vector field is obtained by performing optical flow field calculations on consecutive frames, and the elastic strength index is evaluated based on the displacement vector field. Specifically: ; Where EI represents the elastic strength index, Ω represents the displacement vector field, and Area represents the area of ​​the fruit stem region; The morphological normality of the fruit stem is calculated by morphological analysis of its shape feature parameters and structural integrity index based on skeleton extraction. The fruit stem characteristic evaluation coefficient is constructed by combining green coverage, dry crack density factor, browning index, elastic strength index and morphological incompleteness.

[0011] Preferably, the construction of the fruit stem feature evaluation coefficient includes the following steps: Standardize the comprehensive green coverage, dry crack density factor, browning index, elastic strength index and morphological incompleteness respectively; An improved CRITIC objective weighting method is adopted, which calculates weights based on feature contrast strength and conflict, specifically as follows: ; Where, ω j σ represents the weight of the j-th feature. j Let r represent the standard deviation of the j-th feature. ij The correlation coefficient between features i and j is represented by n, where n represents the total number of features and m represents the total number of samples. The evaluation coefficient for the fruit stem feature is obtained by weighted summation, specifically: ; Where R represents the fruit stem characteristic evaluation coefficient, ω i F represents the weight of the i-th standardized feature parameter. i This represents the standardized feature parameters.

[0012] Preferably, in step S4, the maturity determination model includes the following steps: A dual-channel convolutional neural network with an attention fusion mechanism is used as the basic framework. The dual-channel convolutional neural network includes a feature encoding branch and a feature interaction branch. The feature encoding branch is configured with four parallel convolutional modules, specifically: Color feature module: The color feature matrix is ​​passed through 3 layers of convolution, and then through 2*2 max pooling to output a 256-channel feature map after dimensionality reduction; Fruit powder feature module and fruit stem feature module: The mapped d-dimensional vector is expanded into a 3*3*d matrix, processed by 3 layers of convolution, and then combined with 2*2 max pooling for dimensionality reduction; Shape feature module: Expand the 3D vector into a 1*1*3 matrix, process it through 3 layers of convolution, and then combine it with 2*2 max pooling for dimensionality reduction; The feature interaction branch is as follows: the fruit powder feature and the fruit stem feature interact through a spatial attention mechanism. The spatial attention mechanism calculates the weight of each position in the feature map to highlight the key region features. The shape feature is fused with the color, fruit powder and fruit stem features respectively through residual connections. The output layer outputs a multilayer perceptron after global average pooling of the fused features. This perceptron contains two hidden layers and outputs a probability distribution of four maturity levels. The level corresponding to the highest probability is taken as the result. The four maturity levels include immature, early maturity, medium maturity, and mature.

[0013] Preferably, the criteria for classifying the maturity level are as follows: The immature characteristics are: the green band reflectance ratio of the principal component 1 of the color feature is >90%, 0 < fruit powder feature evaluation coefficient ≤0.4, 0.8 < fruit stem feature evaluation coefficient ≤1, and the diameter of the shape feature is <8mm. The initial ripening stage is defined as follows: 10% ≤ the proportion of reflectance of the red-purple band corresponding to the main component 1 of the color feature ≤ 70%, 0.5 ≤ the fruit powder feature evaluation coefficient ≤ 0.6, 0.6 < the fruit stem feature evaluation coefficient ≤ 0.8, and in the shape feature, 8mm ≤ diameter < 10mm. The medium-ripe state is defined as follows: 50% ≤ the proportion of reflectance of the red-purple band corresponding to the principal component 1 of the color feature ≤ 80%, 0.7 ≤ the fruit powder feature evaluation coefficient ≤ 0.8, 0.4 < the fruit stem feature evaluation coefficient ≤ 0.6, and in the shape feature, 10mm ≤ diameter < 12mm. The maturity is defined as follows: the reflectance of the dark purple band corresponding to the principal component 1 of the color feature is >90%, the evaluation coefficient of the fruit powder feature is >0.8, the evaluation coefficient of the fruit stem feature is ≤0.4, and the diameter is ≤12mm in the shape feature.

[0014] Preferably, the following steps are also included: A distributed database is built based on blockchain technology to store blueberry image features and ripeness labels, and smart contracts are used to define model update rules. When the number of samples on the chain reaches 1000, a federated learning task is triggered: each node trains the model using local data and only uploads the gradient parameters to the chain. The gradient parameters are aggregated using the FedAvg algorithm, verified by digital signature, and then written into a new block for incremental model updates.

[0015] A second aspect of the present invention provides a computer vision-based blueberry ripeness detection system, comprising the following modules: Multispectral acquisition module: Acquires image information of blueberry samples from different angles using a multispectral camera, with the acquisition band covering 400-1000nm; Preprocessing and ROI segmentation module: Denoises, corrects distortion, and segments the regions of interest in the acquired images to obtain individual images of a single blueberry; Multi-dimensional feature extraction module: Based on a preset feature extraction model, it extracts color features, bloom features, stem features, and shape features from an independent image of a single blueberry. The color features are divided into 60 channels based on the 400-700nm wavelength band at 10nm intervals. Principal component analysis is used to retain the first three principal components to form a color feature matrix. The bloom features are comprehensively represented by constructing bloom feature evaluation coefficients. The stem features are comprehensively represented by constructing stem feature evaluation coefficients. The shape features are calculated by measuring their diameter, curvature, and regularity to form a 3D shape feature vector. Maturity Decision Engine Module: The extracted features are input into the trained maturity determination model, and the maturity level of blueberries is output. Among them, the evaluation coefficients of fruit powder and stem are mapped into a vector with the same number of color feature channels through the feature embedding layer, and then input into the model together with other features.

[0016] The beneficial effects of this invention are: This invention comprehensively captures the biological changes in blueberries during the ripening process by integrating four dimensions of features: color, bloom, stem, and shape, thereby improving the accuracy of blueberry ripeness determination. This invention solves the problem of low efficiency in processing heterogeneous data by designing a dedicated encoding module for heterogeneous features (expanding numerical evaluation coefficients into matrices and then inputting them into a convolutional network), and combining spatial attention mechanism and residual connection to strengthen feature association. This invention securely integrates blueberry data from different origins and varieties through a blockchain-based distributed database and federated learning mechanism. After incremental updates, the model's adaptability to new samples is significantly improved. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the module flow of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 As shown, this invention is a computer vision-based method for detecting blueberry ripeness, comprising the following steps: S1: Image information of blueberry samples is acquired from different angles using a multispectral camera, with the acquisition band covering 400-1000nm; S2: Denoise the acquired images, correct distortion, and segment the regions of interest to obtain individual images of a single blueberry; S3: Based on a preset feature extraction model, extract color features, bloom features, stem features, and shape features from an independent image of a single blueberry. The color features are divided into 60 channels based on the 400-700nm wavelength band at 10nm intervals. Principal component analysis is used to retain the first three principal components to form a color feature matrix. The bloom features are comprehensively represented by constructing bloom feature evaluation coefficients. The stem features are comprehensively represented by constructing stem feature evaluation coefficients. The shape features are calculated by measuring their diameter, curvature, and regularity to form a 3D shape feature vector. S4: Input the extracted features into the trained maturity determination model and output the maturity level of blueberries. The evaluation coefficients of fruit powder and stem are mapped to a vector with the same number of color feature channels through the feature embedding layer, and then input into the model together with other features.

[0020] In one embodiment of the present invention, step S2 includes the following steps: Noise is removed by combining Gaussian filtering with nonlocal means denoising algorithm, and distortion correction is performed based on camera intrinsic parameter matrix; The region of interest is segmented by combining deep learning semantic segmentation and adaptive threshold segmentation. Specifically, the blueberry region is initially located using a pre-trained U-Net model, and then the threshold parameters are dynamically adjusted based on the average reflectance of the 650nm band. The Otsu algorithm is then used to segment the individual images of a single blueberry.

[0021] Specifically, a pre-trained U-Net model is loaded, and the distortion-corrected image is input. The model outputs a probability heatmap of the blueberry region, using a probability threshold of 0.5 for the heatmap. Figure Two The blueberry region is initially masked by quantization, with pixels within the mask marked as 1 and the background as 0. The segmentation results are further optimized by considering the spectral characteristics of the 650nm band (a sensitive band for blueberry ripeness): pixels in the 650nm band image within the initial mask are extracted, and their average reflectance 'a' is calculated. The threshold range of the Otsu algorithm is dynamically adjusted based on the average reflectance to [a-30, a+30]. The optimal segmentation threshold is determined using the maximum inter-class variance method. The optimal threshold is applied to the 650nm band image to obtain the fruit region. Connectivity analysis is performed on the optimized mask, and the contours of each connection threshold are extracted using the findoutours function. Effective fruit regions are selected based on the blueberry fruit size characteristics. Each effective region is cropped and its resolution is uniformly adjusted to 256×256 pixels to obtain an independent image of a single blueberry for subsequent feature extraction.

[0022] In one embodiment of the present invention, step S3, the fruit powder characteristics, includes the following steps: An improved U-Net++ network is used to segment the pink region. Specifically, the encoder introduces the inverse residual structure of MobileNetV3, and a spectral attention module is added to the decoder. By combining the segmentation results from both sources, the white powdery area and the fruit body area are segmented using the reflectivity characteristics of the 850nm near-infrared band and the Otsu threshold segmentation results. The percentage of the white bloom area on the blueberry surface is calculated as the white bloom coverage rate of the blueberry. The box dimension is calculated based on fractal geometry as the surface roughness of blueberries; The uniformity of the white bloom area distribution on the blueberry surface is calculated as follows: ; Where U represents uniformity, σ A μ represents the local variance within the sliding window. A This represents the local mean within a sliding window, which is 5x5 pixels in size. The disparity map is generated by combining a stereo matching algorithm, and the 3D point cloud data is reconstructed by triangulation to measure the thickness of the fruit powder layer. Specifically: ; Where h(x,y) represents the thickness of the fruit powder layer at image coordinates (x,y), (x,y) represents the pixel coordinates in the image, and Z... surface Z represents the depth value of the fruit powder surface in three-dimensional coordinates. baseline This represents the depth value of the blueberry fruit's surface in three-dimensional coordinates.

[0023] Specifically, to accurately segment the white bloom region on the surface of blueberries, targeted improvements were made to the U-Net++ network. The specific steps are as follows: The inverse residual structure of MobileNetV3 was introduced to replace the traditional convolutional layers. The inverse residual structure consists of 1*1 convolutions, 3*3 depthwise separable convolutions, and 1*1 convolutions. The dilation rate of the depthwise separable convolutions was set to 1, and ReLU6 was applied. The activation function introduces nonlinearity, reducing the number of parameters by 50% while maintaining feature extraction capability. A spectral attention module is added after each upsampling module of the decoder, designed for the 850nm near-infrared band. This module performs global average pooling on the multi-channel feature map output by the decoder to obtain channel-level feature vectors. Attention weights are calculated through two fully connected layers (the number of neurons in the hidden layer is one-quarter of the number of channels). The attention weights are multiplied by the original feature map to enhance the feature response of the fruit powder-related band. The training set is constructed by collecting 10,000 blueberry images containing fruit powder, manually labeling the fruit powder region as a ground truth mask, and dividing the training set and validation set in an 8:2 ratio. The Adam optimizer is used, and the loss function is Dice loss + cross-entropy loss (weight ratio 1:1). The iteration is performed for 50 rounds until the Dice coefficient of the validation set stabilizes above 0.92. Combining the reflectivity characteristics of the 850nm near-infrared band with Otsu threshold segmentation, the segmentation accuracy of the fruit powder region is further optimized. Specifically, the 850nm band image is extracted from the multispectral image. In this band, the reflectivity of the fruit powder region (60%-80%) is significantly higher than that of the fruit body region (20%=40%). The feature difference is enhanced by contrast enhancement. The Otsu algorithm is applied to the stretched 850nm band image to automatically calculate the optimal threshold and obtain the fruit powder candidate region mask. The preliminary mask output by the improved U-Net++ is then logically ANDed with the Otsu segmentation mask, as shown in the formula: ; Here, & represents a pixel-by-pixel logical AND operation, which removes mis-segmented areas caused by light reflection, and finally obtains accurate segmentation results of the white fruit powder area and the fruit body area; Based on the optimized bloom segmentation mask, the blueberry white bloom coverage, roughness, uniformity, and thickness characteristic parameters are calculated. The blueberry white bloom coverage... , Where, N powder N represents the total number of pixels in the pink area. fruit The total number of pixels in the blueberry fruit is represented by the value 's'. The surface roughness is calculated based on fractal geometry. The powdery area mask is binarized, and boxes of different sizes 's' are used to cover the image. The number of boxes containing powdery pixels, N(S), is counted. The linear relationship between log(N(S)) and log(1 / s) is fitted by the least squares method. The slope is the box dimension, and the value is between 1 and 2, which is the surface roughness. The larger the value, the rougher the surface.

[0024] In one embodiment of the present invention, the construction of fruit powder characteristic evaluation coefficients includes the following steps: The white bloom coverage, surface roughness, uniformity, and bloom layer thickness of blueberries were standardized respectively. The standardized feature parameters were then weighted and summed to construct a bloom characteristic evaluation coefficient, specifically: ; Where K represents the bloom characteristic evaluation coefficient, β1, β2, β3 and β4 represent the dynamic weights of blueberry white bloom coverage, standardized blueberry surface roughness, uniformity and average bloom layer thickness, respectively, and C represents blueberry white bloom coverage. U represents the standardized surface roughness of blueberries, and U represents uniformity. This represents the average bloom thickness. Specifically, let the measured bloom coverage of all blueberries in the sample set be {C1, C2, ..., C...}. n} where n is the number of samples, the maximum and minimum values ​​are determined, and the mapping rate is covered to the [0,1] interval using min-max standardization; the box dimension is known to be in the range of 1.0-2.0, and standardization is performed directly based on the theoretical interval, i.e., by subtracting 1 from the box dimension; the original uniformity calculation result is already in the [0,1] interval, no additional scaling is needed, and the original value is directly retained as the standardization result; the average bloom layer thickness is standardized using the same method as the blueberry white bloom coverage rate; the initial weights are set to an equal weight distribution, and the weights are adjusted by gradient descent, iteratively optimized until the loss function converges, and the final weight values ​​are obtained. The dynamic weights are adaptively adjusted according to the sample distribution. For example, in mature samples, the values ​​of β1, β2, β3 and β4 are 0.2, 0.3, 0.3 and 0.2, respectively.

[0025] In one embodiment of the present invention, step S3, the fruit stem feature, includes the following steps: The UNet++ network is trained to generate a heatmap of the fruit stem region, where P represents the probability heatmap of the fruit stem region, H represents the height of the heatmap, and W represents the width of the heatmap. The improved U-Net image segmentation algorithm is then used to determine the region where the fruit stem is located. The percentage of pixels with H values ​​in the HSV color space between 40° and 80° in the fruit stem region is calculated to obtain its green occupancy, and the dry crack density factor and browning index are extracted. The displacement vector field is obtained by performing optical flow field calculations on consecutive frames, and the elastic strength index is evaluated based on the displacement vector field. Specifically: ; Where EI represents the elastic strength index, Ω represents the displacement vector field, and Area represents the area of ​​the fruit stem region; The morphological normality of the fruit stem is calculated by morphological analysis of its shape feature parameters and structural integrity index based on skeleton extraction. The fruit stem characteristic evaluation coefficient is constructed by combining green coverage, dry crack density factor, browning index, elastic strength index and morphological incompleteness.

[0026] Specifically, 10,000 blueberry images containing stems were collected, and the stem region contours were manually annotated to generate a binarized mask. The training and validation sets were divided in a 7:3 ratio. The U-Net++ network encoder used four convolutional layers (channel count 64→128→256→512), and the decoder used four upsampling layers. The loss function was cross-entropy loss, and the optimizer was Adam (learning rate 0.0001, 50 iterations). Individual blueberry images were input into the trained U-Net++ network to generate a stem region probability heatmap with pixel values ​​ranging from 0 to 255 (higher values ​​indicate a greater probability of a stem). The improved U-Net++ algorithm optimized segmentation by adding an edge enhancement branch after each convolutional module of the U-Net++ decoder and extracting gradient features using the Sobel operator, concatenating them with the original features to improve stem edge segmentation accuracy. The segmentation process binarized the U-Net++ output heatmap with a threshold of 0.6 to obtain a preliminary mask, which was then input into the improved U-Net++ network. The network performs fine segmentation and outputs a precise mask of the fruit stem region. Based on the segmented fruit stem region mask, multiple features are extracted. Morphological skeleton extraction is performed on the fruit stem mask to obtain the fruit stem skeleton line segments. The total length of the skeleton line segments and the longest axis length of the fruit stem region are calculated. The morphological normality is obtained by dividing the total length of the skeleton line segments by the longest axis length of the fruit stem region, and the morphological abnormality is obtained by subtracting the morphological normality from 1. Specifically, the desiccation density factor is: ; Where ρ represents the crack density factor. Let P denote the Laplace operator, and let P denote the probability heatmap of the stem region. Represents the gradient operator; The browning index is specifically: ; Where BI represents the browning index, R 200ne G 200ne and B 200ne These represent the red, green, and blue light component values ​​in the 200nm wavelength band, respectively.

[0027] In one embodiment of the present invention, the construction of the fruit stem feature evaluation coefficient includes the following steps: Standardize the comprehensive green coverage, dry crack density factor, browning index, elastic strength index and morphological incompleteness respectively; An improved CRITIC objective weighting method is adopted, which calculates weights based on feature contrast strength and conflict, specifically as follows: ; Where, ω j σ represents the weight of the j-th feature. j Let r represent the standard deviation of the j-th feature. ij The correlation coefficient between features i and j is represented by n, where n represents the total number of features and m represents the total number of samples. The evaluation coefficient for the fruit stem feature is obtained by weighted summation, specifically: ; Where R represents the fruit stem characteristic evaluation coefficient, ω i F represents the weight of the i-th standardized feature parameter. i This represents the standardized feature parameters.

[0028] Specifically, for green coverage and morphological incompleteness, min-max standardization is used to map to [0,1]. The upper and lower limits of the crack density factor, browning index, and elastic strength index are determined through sample set statistics and standardized to [0,1]. The standard deviation of each feature and the correlation coefficient between features are calculated to obtain the corresponding weights. The values ​​of ω1, ω2, ω3, ω4, and ω5 are 0.3, 0.15, 0.15, 0.2, and 0.2, respectively.

[0029] In one embodiment of the present invention, step S4, the maturity determination model, includes the following steps: A dual-channel convolutional neural network with an attention fusion mechanism is used as the basic framework. The dual-channel convolutional neural network includes a feature encoding branch and a feature interaction branch. The feature encoding branch is configured with four parallel convolutional modules, specifically: Color feature module: The color feature matrix is ​​passed through 3 layers of convolution, and then through 2*2 max pooling to output a 256-channel feature map after dimensionality reduction; Fruit powder feature module and fruit stem feature module: The mapped d-dimensional vector is expanded into a 3*3*d matrix, processed by 3 layers of convolution, and then combined with 2*2 max pooling for dimensionality reduction; Shape feature module: Expand the 3D vector into a 1*1*3 matrix, process it through 3 layers of convolution, and then combine it with 2*2 max pooling for dimensionality reduction; The feature interaction branch is as follows: the fruit powder feature and the fruit stem feature interact through a spatial attention mechanism. The spatial attention mechanism calculates the weight of each position in the feature map to highlight the key region features. The shape feature is fused with the color, fruit powder and fruit stem features respectively through residual connections. The output layer outputs a multilayer perceptron after global average pooling of the fused features. This perceptron contains two hidden layers and outputs a probability distribution of four maturity levels. The level corresponding to the highest probability is taken as the result. The four maturity levels include immature, early ripening, mid-ripening, and mature.

[0030] Specifically, the input is a 3*H*W color feature matrix, where H and W are the height and width of a single blueberry image, uniformly 256*256 pixels; the evaluation coefficients of the fruit powder and stem features map them to d dimensions, where d=3, consistent with the number of color feature channels, expanding into a 3*3*3 three-dimensional matrix; the 3-dimensional shape feature vector is expanded into a 1*1*3 three-dimensional matrix to ensure compatibility with the input format of the convolutional layer; feature fusion is achieved through spatial attention mechanism and residual connection, the specific steps of which are as follows: 1. Spatial attention interaction between fruit powder features and stem features: the fruit powder and stem feature maps are concatenated by channel to obtain a 512×128×128 matrix. The spatial attention module calculation includes: 1. Performing global average pooling and global max pooling on the fused feature map to obtain two 1×128×128 feature maps, which are then concatenated and compressed into a 1×128×128 attention weight map through a 1×1 convolution. The weight map is multiplied pixel-by-pixel with the original fruit powder / stem feature map to highlight key features in the fruit powder shedding area and the fruit stem browning area; 2. Residual connection fusion of shape features: The shape feature map is residually connected with the color, fruit powder, and fruit stem feature maps respectively. The fused feature = main feature map + 0.1 * shape feature map; 3. Cross-feature fusion: The interactive color, fruit powder, and fruit stem features are fused together. The images are stitched together by channel to obtain a final fused feature map of 768×128×128. Global average pooling is then performed on the 768*128*128 fused feature map to obtain a 768-dimensional feature vector. The multilayer perceptron classification includes a first hidden layer with 512 neurons, ReLU activation, and a dropout rate of 0.5; a second hidden layer with 256 neurons, ReLU activation, and a dropout rate of 0.5; and an output layer with 4 neurons, softmax activation, corresponding to the probabilities of four levels: immature, early ripe, medium ripe, and ripe. The level with the highest output probability is taken as the blueberry ripeness detection result.

[0031] In one embodiment of the present invention, the criteria for classifying maturity levels are specifically as follows: The immature characteristics are: the green band reflectance ratio of the principal component 1 of the color feature is >90%, 0 < fruit powder feature evaluation coefficient ≤0.4, 0.8 < fruit stem feature evaluation coefficient ≤1, and the diameter of the shape feature is <8mm. The initial ripening stage is defined as follows: 10% ≤ the proportion of reflectance of the red-purple band corresponding to the main component 1 of the color feature ≤ 70%, 0.5 ≤ the fruit powder feature evaluation coefficient ≤ 0.6, 0.6 < the fruit stem feature evaluation coefficient ≤ 0.8, and in the shape feature, 8mm ≤ diameter < 10mm. The medium-ripe state is defined as follows: 50% ≤ the proportion of reflectance of the red-purple band corresponding to the principal component 1 of the color feature ≤ 80%, 0.7 ≤ the fruit powder feature evaluation coefficient ≤ 0.8, 0.4 < the fruit stem feature evaluation coefficient ≤ 0.6, and in the shape feature, 10mm ≤ diameter < 12mm. The maturity is defined as follows: the reflectance of the dark purple band corresponding to the principal component 1 of the color feature is >90%, the evaluation coefficient of the fruit powder feature is >0.8, the evaluation coefficient of the fruit stem feature is ≤0.4, and the diameter is ≤12mm in the shape feature.

[0032] In one embodiment of the present invention, the following steps are further included: A distributed database is built based on blockchain technology to store blueberry image features and ripeness labels, and smart contracts are used to define model update rules. When the number of samples on the chain reaches 1000, a federated learning task is triggered: each node trains the model using local data and only uploads the gradient parameters to the chain. The gradient parameters are aggregated using the FedAvg algorithm, verified by digital signature, and then written into a new block for incremental model updates.

[0033] Specifically, participating nodes include blueberry plantations, fresh food e-commerce platforms, and testing institutions. Each node is configured with an independent server (CPU ≥ 8 cores, memory ≥ 16GB, storage ≥ 1TB) and installs an Ethereum client (such as Geth) as the blockchain node client. Nodes connect via a P2P network and adopt a PoS (Proof-of-Stake) consensus mechanism to reduce computing power consumption and improve transaction confirmation speed (block time is set to 10 seconds). Each piece of on-chain data contains the following fields: sample ID, image features, maturity label, collection time, collection location, and node signature. The data is stored using the AES-256 encryption algorithm, and only authorized nodes can decrypt and access it using their private keys. After collecting blueberry sample data, each node performs hash calculations to generate data fingerprints. The data and fingerprints are broadcast to the blockchain network through the client. After verification by other nodes, the data is packaged into a new block and associated with the hash value of the previous block to complete on-chain storage. The smart contract collects the gradient parameters uploaded by each node, verifies the signature validity, and then calls the FedAvg algorithm for aggregation. The aggregation process is executed in an on-chain sandbox environment to ensure transparent and traceable computation.

[0034] Please see Figure 2 As shown, this invention is a computer vision-based blueberry ripeness detection system, comprising the following modules: Multispectral acquisition module: Acquires image information of blueberry samples from different angles using a multispectral camera, with the acquisition band covering 400-1000nm; Preprocessing and ROI segmentation module: Denoises, corrects distortion, and segments the regions of interest in the acquired images to obtain individual images of a single blueberry; Multi-dimensional feature extraction module: Based on a preset feature extraction model, it extracts color features, bloom features, stem features, and shape features from an independent image of a single blueberry. The color features are divided into 60 channels based on the 400-700nm wavelength band at 10nm intervals. Principal component analysis is used to retain the first three principal components to form a color feature matrix. The bloom features are comprehensively represented by constructing bloom feature evaluation coefficients. The stem features are comprehensively represented by constructing stem feature evaluation coefficients. The shape features are calculated by measuring their diameter, curvature, and regularity to form a 3D shape feature vector. Maturity Decision Engine Module: The extracted features are input into the trained maturity determination model, and the maturity level of blueberries is output. Among them, the evaluation coefficients of fruit powder and stem are mapped into a vector with the same number of color feature channels through the feature embedding layer, and then input into the model together with other features.

[0035] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A computer vision-based method for detecting blueberry ripeness, characterized in that, Includes the following steps: S1: Image information of blueberry samples is acquired from different angles using a multispectral camera, with the acquisition band covering 400-1000nm; S2: Denoise the acquired images, correct distortion, and segment the regions of interest to obtain individual images of a single blueberry; S3: Based on a preset feature extraction model, extract color features, bloom features, stem features, and shape features from an independent image of a single blueberry. The color features are divided into 60 channels based on the 400-700nm wavelength band at 10nm intervals. Principal component analysis is used to retain the first three principal components to form a color feature matrix. The bloom features are comprehensively represented by constructing bloom feature evaluation coefficients. The stem features are comprehensively represented by constructing stem feature evaluation coefficients. The shape features are calculated by measuring their diameter, curvature, and regularity to form a 3D shape feature vector. S4: Input the extracted features into the trained maturity determination model and output the maturity level of blueberries. The evaluation coefficients of fruit powder and stem are mapped to a vector with the same number of color feature channels through the feature embedding layer, and then input into the model together with other features.

2. The blueberry ripeness detection method based on computer vision according to claim 1, characterized in that, Step S2 includes the following steps: Noise is removed by combining Gaussian filtering with nonlocal means denoising algorithm, and distortion correction is performed based on camera intrinsic parameter matrix; The region of interest is segmented by combining deep learning semantic segmentation and adaptive threshold segmentation. Specifically, the blueberry region is initially located using a pre-trained U-Net model, and then the threshold parameters are dynamically adjusted based on the average reflectance of the 650nm band. The Otsu algorithm is then used to segment the individual images of a single blueberry.

3. The blueberry ripeness detection method based on computer vision according to claim 1, characterized in that, In step S3, the fruit powder characteristics include the following steps: An improved U-Net++ network is used to segment the pink region. Specifically, the encoder introduces the inverse residual structure of MobileNetV3, and a spectral attention module is added to the decoder. By combining the segmentation results from both sources, the white powdery area and the fruit body area are segmented using the reflectivity characteristics of the 850nm near-infrared band and the Otsu threshold segmentation results. The percentage of the white bloom area on the blueberry surface is calculated as the white bloom coverage rate of the blueberry. The box dimension is calculated based on fractal geometry as the surface roughness of blueberries; The uniformity of the white bloom area distribution on the blueberry surface is calculated as follows: ; Where U represents uniformity, σ A μ represents the local variance within the sliding window. A This represents the local mean within a sliding window, which is 5x5 pixels in size. The disparity map is generated by combining a stereo matching algorithm, and the 3D point cloud data is reconstructed by triangulation to measure the thickness of the fruit powder layer. Specifically: ; Where h(x,y) represents the thickness of the fruit powder layer at image coordinates (x,y), (x,y) represents the pixel coordinates in the image, and Z... surface Z represents the depth value of the fruit powder surface in three-dimensional coordinates. baseline This represents the depth value of the blueberry fruit's surface in three-dimensional coordinates.

4. The blueberry ripeness detection method based on computer vision according to claim 1, characterized in that, The construction of the fruit powder characteristic evaluation coefficient includes the following steps: The white bloom coverage, surface roughness, uniformity, and bloom layer thickness of blueberries were standardized respectively. The standardized feature parameters were then weighted and summed to construct a bloom characteristic evaluation coefficient, specifically: ; Where K represents the bloom characteristic evaluation coefficient, β1, β2, β3 and β4 represent the dynamic weights of blueberry white bloom coverage, standardized blueberry surface roughness, uniformity and average bloom layer thickness, respectively, and C represents blueberry white bloom coverage. U represents the standardized surface roughness of blueberries, and U represents uniformity. This indicates the average thickness of the bloom layer.

5. The blueberry ripeness detection method based on computer vision according to claim 1, characterized in that, In step S3, the fruit stem characteristics include the following steps: Heatmap of the result area generated by training the UNet++ network Where P represents the probability heatmap of the fruit stem region, H represents the height of the heatmap, and W represents the width of the heatmap. The improved U-Net image segmentation algorithm is used to determine the region where the fruit stem is located. The percentage of pixels with H values ​​between 40° and 80° in the HSV color space within the fruit stem region is calculated to obtain its green occupancy. The crack density factor and browning index are also extracted. Optical flow field calculations are performed on consecutive frames to obtain the displacement vector field. The elastic strength index is then evaluated based on the displacement vector field. ; Where EI represents the elastic strength index, Ω represents the displacement vector field, and Area represents the area of ​​the fruit stem region; The morphological normality of the fruit stem is calculated by morphological analysis of its shape feature parameters and structural integrity index based on skeleton extraction. The fruit stem characteristic evaluation coefficient is constructed by combining green coverage, dry crack density factor, browning index, elastic strength index and morphological incompleteness.

6. The blueberry ripeness detection method based on computer vision according to claim 5, characterized in that, The construction of the fruit stem feature evaluation coefficient includes the following steps: Standardize the comprehensive green coverage, dry crack density factor, browning index, elastic strength index and morphological incompleteness respectively; An improved CRITIC objective weighting method is adopted, which calculates weights based on feature contrast strength and conflict, specifically as follows: ; Where, ω j σ represents the weight of the j-th feature. j Let r represent the standard deviation of the j-th feature. ij The correlation coefficient between features i and j is represented by n, where n represents the total number of features and m represents the total number of samples. The evaluation coefficient for the fruit stem feature is obtained by weighted summation, specifically: ; Where R represents the fruit stem characteristic evaluation coefficient, ω i F represents the weight of the i-th standardized feature parameter. i This represents the standardized feature parameters.

7. The blueberry ripeness detection method based on computer vision according to claim 1, characterized in that, In step S4, the maturity determination model includes the following steps: A dual-channel convolutional neural network with an attention fusion mechanism is used as the basic framework. The dual-channel convolutional neural network includes a feature encoding branch and a feature interaction branch. The feature encoding branch is configured with four parallel convolutional modules, specifically: Color feature module: The color feature matrix is ​​passed through 3 layers of convolution, and then through 2*2 max pooling to output a 256-channel feature map after dimensionality reduction; Fruit powder feature module and fruit stem feature module: The mapped d-dimensional vector is expanded into a 3*3*d matrix, processed by 3 layers of convolution, and then combined with 2*2 max pooling for dimensionality reduction; Shape feature module: Expand the 3D vector into a 1*1*3 matrix, process it through 3 layers of convolution, and then combine it with 2*2 max pooling for dimensionality reduction; The feature interaction branch is as follows: the fruit powder feature and the fruit stem feature interact through a spatial attention mechanism. The spatial attention mechanism calculates the weight of each position in the feature map to highlight the key region features. The shape feature is fused with the color, fruit powder and fruit stem features respectively through residual connections. The output layer outputs a multilayer perceptron after global average pooling of the fused features. This perceptron contains two hidden layers and outputs a probability distribution of four maturity levels. The level corresponding to the highest probability is taken as the result. The four maturity levels include immature, early ripening, mid-ripening, and mature.

8. The blueberry ripeness detection method based on computer vision according to claim 7, characterized in that, The criteria for classifying maturity levels are as follows: The immature characteristics are: the green band reflectance ratio of the principal component 1 of the color feature is >90%, 0 < fruit powder feature evaluation coefficient ≤0.4, 0.8 < fruit stem feature evaluation coefficient ≤1, and the diameter of the shape feature is <8mm. The initial ripening stage is defined as follows: 10% ≤ the proportion of reflectance of the red-purple band corresponding to the main component 1 of the color feature ≤ 70%, 0.5 ≤ the fruit powder feature evaluation coefficient ≤ 0.6, 0.6 < the fruit stem feature evaluation coefficient ≤ 0.8, and in the shape feature, 8mm ≤ diameter < 10mm. The medium-ripe state is defined as follows: 50% ≤ the proportion of reflectance of the red-purple band corresponding to the principal component 1 of the color feature ≤ 80%, 0.7 ≤ the fruit powder feature evaluation coefficient ≤ 0.8, 0.4 < the fruit stem feature evaluation coefficient ≤ 0.6, and in the shape feature, 10mm ≤ diameter < 12mm. The maturity is defined as follows: the reflectance of the dark purple band corresponding to the principal component 1 of the color feature is >90%, the evaluation coefficient of the fruit powder feature is >0.8, the evaluation coefficient of the fruit stem feature is ≤0.4, and the diameter is ≤12mm in the shape feature.

9. The blueberry ripeness detection method based on computer vision according to claim 1, characterized in that, It also includes the following steps: A distributed database is built based on blockchain technology to store blueberry image features and ripeness labels, and smart contracts are used to define model update rules. When the number of samples on the chain reaches 1000, a federated learning task is triggered: each node trains the model using local data and only uploads the gradient parameters to the chain. The gradient parameters are aggregated using the FedAvg algorithm, verified by digital signature, and then written into a new block for incremental model updates.

10. A computer vision-based blueberry ripeness detection system, characterized in that, Includes the following modules: Multispectral acquisition module: Acquires image information of blueberry samples from different angles using a multispectral camera, with the acquisition band covering 400-1000nm; Preprocessing and ROI segmentation module: Denoises, corrects distortion, and segments the regions of interest in the acquired images to obtain individual images of a single blueberry; Multi-dimensional feature extraction module: Based on a preset feature extraction model, it extracts color features, bloom features, stem features, and shape features from an independent image of a single blueberry. The color features are divided into 60 channels based on the 400-700nm wavelength band at 10nm intervals. Principal component analysis is used to retain the first three principal components to form a color feature matrix. The bloom features are comprehensively represented by constructing bloom feature evaluation coefficients. The stem features are comprehensively represented by constructing stem feature evaluation coefficients. The shape features are calculated by measuring their diameter, curvature, and regularity to form a 3D shape feature vector. Maturity Decision Engine Module: The extracted features are input into the trained maturity determination model, and the maturity level of blueberries is output. Among them, the evaluation coefficients of fruit powder and stem are mapped into a vector with the same number of color feature channels through the feature embedding layer, and then input into the model together with other features.

Images