Multi-element fusion intelligent fruit preservation monitoring method

Abstract

The invention belongs to the field of postharvest preservation of fruits and vegetables, and particularly discloses a multi-element fusion intelligent fruit preservation monitoring method which comprises the following steps: extracting spectral characteristics of fruits; extracting spatial features of the fruits; environment parameters are collected; through a dynamic main feature selection module, according to the spectrum, spatial features and environmental parameters of the fruits, determining the types of the fruits, querying a fruit feature library, and automatically configuring core detection indexes; carrying out weighted fusion on the spectral feature data and the spatial feature data of the fruits through a bilinear convolutional network, and outputting fusion feature vectors of the maturity and the freshness of the fruits; coupling the fusion feature vectors of the maturity and freshness of the fruits; predicting the remaining shelf life of the fruits according to a maturity-freshness coupling model; and outputting a fruit grading decision and a shelf life early warning signal. According to the method, high compatibility and accurate maturity identification of multiple fruit varieties can be realized, and the evaluation precision of the comprehensive quality of the fruits and the prediction robustness in a complex environment are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of post-harvest preservation of fruits and vegetables, and specifically discloses a multi-integrated intelligent fruit preservation monitoring method. Background Art

[0002] As a major agricultural country, my country consistently ranks first in the world in fruit production. However, issues such as inaccurate monitoring, untimely application of preservation measures, and improper use of preservation methods lead to significant post-harvest fruit losses. To transform fruit from primary raw materials into high-value-added commodities, screening and sorting fruit based on its maturity and freshness during harvesting, packaging, storage, transportation, and post-processing is crucial to protect and improve fruit quality and increase economic benefits.

[0003] Traditional testing techniques rely primarily on manual judgment of the fruit's appearance to identify its maturity and freshness. This is subject to significant influence from subjective factors and instrumentation, resulting in low efficiency and a high rate of misjudgment. With the advancement of science and technology, non-destructive testing technologies, such as electrical property analysis, electronic nose technology, computer vision, near-infrared spectroscopy, and hyperspectral imaging, have attracted significant attention due to their speed, simplicity, and accuracy.

[0004] However, these detection methods are often used singly and have limitations. Traditional monitoring models are designed for specific fruits, making them unsuitable for scenarios with a wide variety of fruit and lacking versatility. Furthermore, traditional methods often assess ripeness and freshness separately or prioritize one over the other, resulting in poor accuracy and prone to misjudgment.

[0005] Based on the above problems, it is urgent to develop a new fruit insurance monitoring method to meet the needs of use. Summary of the Invention

[0006] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a multi-integrated intelligent fruit preservation monitoring method that can achieve high compatibility and accurate identification of maturity of multiple fruit varieties, significantly improve the assessment accuracy of the comprehensive quality of fruits and the prediction robustness in complex environments, and effectively solve the problems in the background technology.

[0007] The present invention is achieved through the following technical solutions:

[0008] A multi-faceted intelligent fruit preservation monitoring method, the specific steps are as follows:

[0009] S1. Collect spectral data of fruits through a hyperspectral imaging system and extract spectral characteristics of fruits;

[0010] S2, collect 3D point cloud data of the fruit surface morphology through RGB-D camera and extract the spatial features of the fruit;

[0011] S3, using temperature and humidity sensors to monitor the fruit storage location in real time and collect environmental parameters;

[0012] S4, through the dynamic main feature selection module, according to the fruit characteristics collected by S1 and S2 and the environmental parameters collected by S3, determines the fruit type, queries the fruit feature library, and automatically configures the core detection indicators;

[0013] S5, based on the core detection indicators of the fruit determined in S4, weightedly fuses the spectral features and spatial feature data of the fruit collected by S1 and S2 through a bilinear convolutional network, and outputs a fused feature vector of the maturity and freshness of the fruit;

[0014] S6, using the coupled correlation model, couples the fusion feature vectors of the maturity and freshness of the fruit output by S5 to establish a dynamic relationship between maturity and freshness;

[0015] S7, using the shelf life prediction model, based on the maturity-freshness dynamic relationship coupled with S6, predict the remaining shelf life of the fruit;

[0016] S8, based on the remaining shelf life of the fruit predicted by S7, outputs fruit grading decision and shelf life warning signal;

[0017] Wherein, the bilinear convolutional network is arranged in the dynamic main feature selection module.

[0018] A further improvement of the present invention is:

[0019] In S1, the acquisition band of the hyperspectral imaging system is 400-1000nm.

[0020] A further improvement of the present invention is:

[0021] In S4, the fruit feature library is as follows:

[0022] Fruit Core Features Weight Auxiliary Features apple hardness 0.8 Sugar content, color strawberry Surface depression area 0.9 color Cherry Tomato color 0.85 hardness watermelon Texture complexity 0.75 Sugar content, color

[0023] Furthermore, the fruit feature library is provided with feature weight configuration matrices for a variety of fruits, specifically as follows:

[0024]

[0025] Where m: number of fruit varieties; n: number of characteristic indicators; a ij : The jth weight of the i-th fruit

[0026] A further improvement of the present invention is:

[0027] In S5, the bilinear convolutional network is specifically as follows:

[0028] (1) Spectral branch processing flow:

[0029]

[0030] Where H: hyperspectral data cube; K, W: spatial dimensions; B: number of spectral bands (120 bands); Q: 128-dimensional feature vector;

[0031] (2) Spatial branch processing flow:

[0032]

[0033] Where, ν: RGB-D four-channel image; X: 256-dimensional feature vector;

[0034] (3) Bilinear fusion formula:

[0035]

[0036] Where S: the number of bits in the feature map space; Outer product operation; F: 32768-dimensional fusion feature; (4) Global pooling:

[0037] Bilinear pooling:

[0038]

[0039] Spatial Average Pooling:

[0040]

[0041] Furthermore, the bilinear convolutional network is optimized as follows:

[0042] (1) Feature dimensionality reduction:

[0043]

[0044] Retain the first 512 singular values ​​through SVD;

[0045] (2) Loss function:

[0046]

[0047] Among them, CE: cross entropy loss; α = 0.7, β = 0.3: balance weight.

[0048] A further improvement of the present invention is:

[0049] In S6, the coupling association model is specifically as follows:

[0050] (1) Maturity Quantification Model:

[0051]

[0052] Among them, φ k : kth biochemical index detection function; γ k : Index weight (∑γ k =1);

[0053] (2) Freshness decay equation:

[0054]

[0055] Where, λ: Fruit decay rate constant; E a : activation energy of corruption; R g : gas constant;

[0056] The dynamic relationship between maturity and freshness is realized by the state transition matrix, as follows:

[0057]

[0058] Among them, a 11 : maturity self-attenuation coefficient; a 12 :The effect of freshness on maturity; a 21 :The effect of maturity on freshness; a 22 : Freshness self-attenuation coefficient.

[0059] Furthermore, the coupling correlation model introduces an environmental synergistic factor model and uses a direction vector to regulate the sensitivity differences of fruits, as follows:

[0060] (1) Environmental synergistic factor model:

[0061] Ψ(T,H)=exp(σ T ΔT)·(1+ω H ΔH)·ΔT·ΔH

[0062] Where, ΔT: temperature offset, value range: -10℃~40℃; ΔH: humidity offset, value range: 20%-100%; σ T : Temperature nonlinear coefficient, 0.02~0.15; ω H : Humidity linear coefficient, 0.01~0.10;

[0063] (2) Maturity-freshness coupling correlation model after introducing the environmental synergistic factor model:

[0064]

[0065] Where M: maturity-freshness dynamic coupling state transfer matrix; θ: environmental sensitivity coefficient, 0.8-1.2; l: environmental action direction vector; l R: weight of the impact of environment on maturity; l F : The weight of the environment's impact on freshness.

[0066] A further improvement of the present invention is:

[0067] In S7, the shelf prediction model is specifically as follows:

[0068] (1) Time series input:

[0069]

[0070] Where, τ: time window length: the default value is 24 hours.

[0071] (2) Temporal Convolutional Network (TCN) processing:

[0072] H t =TCN(X t-τ:t )

[0073] (3) Weibull distribution parameter prediction:

[0074] λ=W λ H t +b λ

[0075] k=softplus(W k H t +b k )

[0076] Among them, softplus ensures k>0;

[0077] (4) Weibull distribution prediction:

[0078]

[0079] Among them, λ: scale parameter; k: shape parameter; λ and k are both predicted by TCN.

[0080] Furthermore, a three-stage training strategy is adopted to complete the shelf prediction model training, as follows:

[0081] (1) Basic model: pre-trained using the Apple dataset;

[0082] (2) Domain adaptation: adversarial training is adapted to other varieties of fruit;

[0083] (3) Online fine-tuning: real-time data collection and update.

[0084] Beneficial effects:

[0085] Compared with the prior art, the present invention has the following obvious advantages:

[0086] 1. The present invention provides a multi-integrated intelligent fruit preservation monitoring method. Through the dynamic configuration mechanism of feature weights, it automatically adapts to the differentiated characteristics of different fruit types (such as apples, cherry tomatoes, watermelons, etc.), overcomes the limitation of poor universality of traditional models, and achieves compatibility with multiple fruit types; at the same time, it uses bilinear feature interaction to deeply mine and fuse complex features (such as texture, color, shape, spectrum, etc.), significantly improving the accuracy of distinguishing fruits with similar visual features but different maturity, breaking through the technical bottleneck of inaccurate recognition of similar maturity, and laying the foundation for realizing large-scale, multi-variety automatic grading in orchards.

[0087] Second, the method provided by the present invention couples maturity (reflecting the physiological stage) and freshness (reflecting the post-harvest state) to construct a judgment model, deeply exploring the inherent relationship and mutual influence between the two (for example, the freshness of high-maturity fruits decreases faster). This coupling mechanism can more comprehensively and accurately reflect the overall quality and marketability of fruits, significantly reducing the risk of mis-picking, misclassification, and mis-selling caused by single-indicator evaluation. At the same time, it provides a dynamic basis that is more in line with physiological laws for predicting the optimal consumption window and estimating the remaining shelf life, surpassing the limitations of static evaluation.

[0088] 3. The method provided by the present invention realizes the joint modeling of temperature and humidity interference through environmental synergistic factors, uses direction vectors to regulate the sensitivity differences of fruit varieties, and forms a tightly coupled compensation mechanism, so that the model can maintain a high prediction accuracy in complex environments such as cold chain breaks and cross-border transportation. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 It is a framework diagram of the present invention;

[0090] Figure 2 is a flow chart of the present invention;

[0091] Figure 3 Flowchart of the dynamic main feature selection module;

[0092] Figure 4 Flowchart of the bilinear convolutional network architecture;

[0093] Figure 5 Flowchart of the maturity-freshness coupling correlation model;

[0094] Figure 6 This is a flowchart of the shelf life prediction model;

[0095] Figure 7 These are some experimental apple images and spectral curves collected by the hyperspectral imaging system in Example 1 of the present invention. DETAILED DESCRIPTION

[0096] The present invention is described in detail below with reference to the embodiments.

[0097] Example 1: Apple Freshness Monitoring and Shelf Life Prediction

[0098] 1. Experimental subjects:

[0099] 1,000 Red Fuji apples (single fruit weight 180-220g) produced in Yantai, Shandong were selected and placed in an intelligent monitoring warehouse within 24 hours after harvesting.

[0100] The initial environment settings are 15°C ± 0.5°C and 85% ± 3% RH, simulating cold chain transportation conditions (temperature fluctuation: 15°C → 18°C).

[0101] 2. Experimental process:

[0102] (1) Data collection:

[0103] Spectral data were collected by a hyperspectral imaging system (400–1000 nm) to generate a spectral cube of 256 × 256 × 120 pixels;

[0104] The RGB-D camera captures the 3D point cloud of the surface and detects that the average depth of surface depressions is 0.2 mm ± 0.05;

[0105] Temperature and humidity sensors record environmental fluctuations in real time.

[0106] (2) Feature processing and modeling:

[0107] The dynamic main feature selection module automatically configures the core indicator: hardness (weight 0.8), supplemented by sugar content and color auxiliary features;

[0108] The bilinear convolutional network fuses spectral and spatial features and reduces the dimension to 512 by SVD;

[0109] The maturity-freshness coupling model outputs the state transition matrix:

[0110] (The self-attenuation coefficient dominates the dynamic relationship).

[0111] (3) Forecasting and decision-making:

[0112] The shelf life model (TCN + Weibull distribution) predicted a remaining shelf life of 15.3 days (95% confidence interval 14.8-16.1 days);

[0113] The system outputs the grading results: 82% for first-grade fruit, 18% for second-grade fruit, and issues an early warning for three fruits at risk of corruption.

[0114] 3. Experimental results:

[0115] The accuracy of maturity recognition has been improved. The bilinear network has an accuracy of 96.4% for grading apples of similar color, with an error of <3%. The measured average shelf life is 15.8 days, and the prediction error is ≤3.2%, which improves the reliability of shelf life prediction. Under temperature and humidity fluctuations (15℃, 80% RH), direction vector compensation reduces the prediction error by 32% (from 8.1% to 5.5%).

[0116] Conclusion: This example demonstrates the stability of the multi-feature fusion model in complex environments. The shelf life prediction accuracy is significantly better than the traditional empirical method (error 15%-20%), providing an accurate basis for apple supply chain decision-making.

[0117] Example 2: Multi-variety compatibility verification (strawberry and watermelon)

[0118] 1. Experimental subjects:

[0119] Strawberries (500 pieces, variety "Zhangji") and watermelons (200 pieces, variety "Zaochun Hongyu") are stored in separate areas and mixed.

[0120] Zoning environment settings: strawberry area 8℃±1℃, watermelon area 10℃±1℃, humidity 90%±5%RH.

[0121] 2. Experimental process:

[0122] (1) Data collection:

[0123] Spectral data were collected by a hyperspectral imaging system (400–1000 nm) to generate a spectral cube of 256 × 256 × 120 pixels;

[0124] The RGB-D camera captured the 3D point cloud of the surface and detected an average depth of 0.35mm±0.08 on the strawberry surface and a fractal dimension of 1.68±0.15 on the watermelon texture.

[0125] Temperature and humidity sensors record environmental fluctuations in real time.

[0126] (2) Feature processing and modeling:

[0127] Multi-variety adaptive processing: Dynamic feature library automatically switches core indicators: Strawberry → surface depression area (weight 0.9), watermelon → texture complexity (weight 0.75);

[0128] Bilinear network classification confidence: strawberry 98.7%, watermelon texture extraction error <2%.

[0129] Coupling modeling and environmental compensation: synergy factor compensation value ψ = (0.12, 0.9);

[0130] The capture rate of the maturity-freshness dynamic association reached 89% (the freshness decay rate of high-maturity strawberries was 2.1 times faster than that of the low-maturity group).

[0131] (3) Forecasting and decision-making:

[0132] Full-chain decision optimization:

[0133] Shelf life prediction: See Table 1 below.

[0134] Table 1 Shelf life prediction results

[0135] strawberry watermelon Actual measurement / day 6.2 21.5 Forecast / day 6.0 22.1 error / % 3.2 2.8

[0136] Based on the forecast, the strawberry zone was adjusted to 5°C, extending the shelf life by 18%; the output spoilage warning rate was 12% (mainly due to mechanical damage).

[0137] 3. Experimental results:

[0138] The feature weight switching time takes <0.5s, and the variety recognition accuracy is 100%; the first-grade fruit rate of watermelons reaches 95%, and the strawberry loss rate is reduced by 22%; the execution delay of environmental control commands is <5min, and the user interface data update delay is ≤3s.

[0139] Conclusion: This example verifies the high compatibility of the system in a mixed fruit variety scenario. Through dynamic feature configuration and real-time environmental compensation, it significantly improves the shelf life prediction accuracy and commercialization rate.

[0140] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A multi-integrated intelligent fruit preservation monitoring method, characterized in that: The specific steps are as follows: S1. Collect spectral data of fruits through a hyperspectral imaging system and extract spectral characteristics of fruits; S2, collect 3D point cloud data of the fruit surface morphology through RGB-D camera and extract the spatial features of the fruit; S3, using temperature and humidity sensors to monitor the fruit storage location in real time and collect environmental parameters; S4, through the dynamic main feature selection module, according to the fruit characteristics collected by S1 and S2 and the environmental parameters collected by S3, determines the fruit type, queries the fruit feature library, and automatically configures the core detection indicators; S5, based on the core detection indicators of the fruit determined in S4, weightedly fuses the spectral and spatial feature data of the fruit collected by S1 and S2 through a bilinear convolutional network, and outputs a fused feature vector of the maturity and freshness of the fruit; S6, using the coupled correlation model, couples the fusion feature vectors of the maturity and freshness of the fruit output by S5 to establish a dynamic relationship between maturity and freshness; S7, using the shelf life prediction model, based on the maturity-freshness dynamic relationship coupled with S6, predict the remaining shelf life of the fruit; S8: Output fruit grading decision and shelf life warning signal based on the remaining shelf life of the fruit predicted by S7.

2. The multi-integrated intelligent fruit preservation monitoring method according to claim 1, characterized in that: In S1, the acquisition band of the hyperspectral imaging system is 400-1000nm.

3. The multi-faceted intelligent fruit preservation monitoring method according to claim 1 is characterized by: In S4, the fruit feature library is as follows:

4. The multi-integrated intelligent fruit preservation monitoring method according to claim 3 is characterized by: The fruit feature library is provided with feature weight configuration matrices for various fruits, specifically as follows: Where m: number of fruit varieties; n: number of characteristic indicators; a ij : The jth weight of the i-th fruit 5. The multi-integrated intelligent fruit preservation monitoring method according to claim 1 is characterized by: In S5, the bilinear convolutional network is specifically as follows: (1) Spectral branch processing flow: Q=f 3Dconv (Η), Where H: hyperspectral data cube; K, W: spatial dimensions; B: number of spectral bands (120 bands); Q: 128-dimensional feature vector; (2) Spatial branch processing flow: X=f 2Dconv (n), Where, ν: RGB-D four-channel image; X: 256-dimensional feature vector; (3) Bilinear fusion formula: Where S: the number of bits in the feature map space; Outer product operation; F: 32768-dimensional fusion features; (4) Global pooling: Bilinear pooling: Spatial Average Pooling:

6. The multi-integrated intelligent fruit preservation monitoring method according to claim 5, characterized in that: The bilinear convolutional network is optimized as follows: (1) Feature Dimensionality Reduction: k＝512 Retain the first 512 singular values ​​through SVD; (2) Loss function: Among them, CE: cross entropy loss; α = 0.7, β = 0.3: balance weight.

7. The multi-integrated intelligent fruit preservation monitoring method according to claim 1, characterized in that: In S6, the coupling association model is specifically as follows: (1) Maturity Quantification Model: Among them, φ k : kth biochemical index detection function; γ k : indicator weight (∑γk=1); (2) Freshness decay equation: Where, λ: Fruit decay rate constant; E a : activation energy of corruption; R g : gas constant; The dynamic relationship between maturity and freshness is realized by the state transition matrix, as follows: Among them, a 11 : maturity self-attenuation coefficient; a 12 :The effect of freshness on maturity; a 21 :The influence of maturity on freshness; a 22 : Freshness self-attenuation coefficient.

8. The multi-integrated intelligent fruit preservation monitoring method according to claim 7, characterized in that: The coupling correlation model introduces the environmental synergistic factor model and uses the direction vector to regulate the sensitivity differences of fruits, as follows: (1) Environmental synergistic factor model: Ψ(T,H)=exp(σ T ΔT)·(1+ω H ΔH)·ΔT·ΔH Where, ΔT: temperature offset, value range: -10℃-40℃; ΔH: humidity offset, value range: 20%-100%; σ T : Temperature nonlinear coefficient, 0.02-0.15; ω H : Humidity linear coefficient, 0.01-0.10; (2) Maturity-freshness coupling correlation model after introducing the environmental synergistic factor model: Where M: maturity-freshness dynamic coupling state transfer matrix; θ: environmental sensitivity coefficient, 0.8-1.2; l: environmental action direction vector; l R : weight of the impact of environment on maturity; l F : The weight of the environment's impact on freshness.

9. The multi-integrated intelligent fruit preservation monitoring method according to claim 1, characterized in that: In S7, the shelf prediction model is specifically as follows: (1) Time series input: Where, τ: time window length: default is 24h; (2) Temporal Convolutional Network Processing: H t =BC(X t-τ:t ) (3) Weibull distribution parameter prediction: λ=W λ H t +b λ k=softplus(W k H t +b k ) Among them, softplus ensures k>0; (4) Weibull distribution prediction: Among them, λ: scale parameter; k: shape parameter; λ and k are both predicted by the temporal convolutional network.

10. The multi-integrated intelligent fruit preservation monitoring method according to claim 9, characterized in that: The shelf prediction model training is completed using a three-stage training strategy, as follows: (1) Basic model: pre-trained using the Apple dataset; (2) Domain adaptation: adversarial training is adapted to other varieties of fruit; (3) Online fine-tuning: real-time data collection and update.

Images