Method for predicting change trend of grain quality in granary based on multi-modal data fusion

Abstract

The invention discloses a method for predicting the change trend of grain quality in a granary based on multi-modal data fusion, and relates to the technical field of granary supervision, and the method comprises the steps: judging a grain quality basic index and a grain quality change rate according to preprocessed multi-modal data; performing spatial-temporal feature fusion analysis based on the preprocessed multi-modal data, the grain quality basic index and the grain quality change rate, and constructing a grain quality prediction feature space; generating a grain quality change trend prediction model by using a preset grain quality evolution law and the grain quality prediction feature space; the preset grain quality evolution rule is obtained based on historical multi-modal data and historical granary environment parameters; obtaining current granary environment parameters, and inputting the current granary environment parameters into the grain quality change trend prediction model to output a grain quality change trend prediction result; the demand of intelligent warehouse management is met, and the accuracy and practicability of grain quality change trend prediction are improved.

Description

Technical Field

[0001] This application relates to the field of grain warehouse monitoring technology, and in particular to a method for predicting the trend of grain quality changes in grain warehouses through multimodal data fusion. Background Technology

[0002] In the field of grain storage management, predicting the trend of grain quality changes is a key link in ensuring grain storage safety and optimizing inventory management.

[0003] Existing technologies mostly rely on structured data such as temperature and humidity to build predictive models, neglecting the fusion and utilization of multimodal data such as grain pile images and insect infestation videos. For example, while existing solutions can monitor changes in temperature and humidity in grain silos through sensors, they cannot identify areas of heat generation in the grain pile, signs of insect infestation, or the degree of grain clumping through visual data. This leads to one-sided prediction results that fail to fully reflect the complexity of changes in grain quality.

[0004] Therefore, the shortcomings of existing technologies are that the existing prediction models are not sensitive enough to the deterioration of grain quality, have significant prediction lag, and cannot meet the needs of intelligent warehouse management, so improvements are needed. Summary of the Invention

[0005] To meet the needs of intelligent warehouse management and improve the accuracy and practicality of predicting grain quality change trends, this application provides a method for predicting grain quality change trends in grain warehouses using multimodal data fusion.

[0006] Firstly, the objective of this invention is achieved through the following technical solution: A method for predicting the trend of grain quality changes in granaries by multimodal data fusion includes: Acquire multimodal data from within the grain warehouse and perform preprocessing; Based on the preprocessed multimodal data, determine the basic indicators of grain quality and the rate of change of grain quality; Based on the preprocessed multimodal data, the basic indicators of grain quality, and the rate of change of grain quality, a spatiotemporal feature fusion analysis is performed to construct a grain quality prediction feature space. A grain quality change trend prediction model is generated by using the preset grain quality evolution law and the grain quality prediction feature space; the preset grain quality evolution law is obtained based on historical multimodal data and historical grain storage environmental parameters. Obtain the current grain storage environment parameters, and input the current grain storage environment parameters into the grain quality change trend prediction model to output the grain quality change trend prediction result.

[0007] By adopting the above technical solution, and utilizing multimodal data fusion and spatiotemporal feature extraction, the accuracy and practicality of predicting grain quality change trends within grain warehouses are improved. Specifically, the multimodal data includes structured data, sensor data, and visual data; a spatiotemporally coupled grain quality prediction feature space is constructed using multimodal data to enhance the ability to perceive multidimensional changes in grain quality trends within grain warehouses. By using spatiotemporal feature fusion and grain quality evolution laws to predict the temporal dependence and spatial heterogeneity of grain quality evolution, compared to single-sensor prediction methods, this application can improve the accuracy of grain quality grade prediction, which is beneficial to meeting the needs of intelligent warehouse management.

[0008] In a preferred embodiment of this application: the multimodal data includes structured data, sensor data, and visual data; the acquisition and preprocessing of multimodal data within the grain silo includes: Based on the grain depot equipment deployment information, a multimodal data collection list was determined; The validity of the collected raw multimodal data is verified based on the preset data quality assessment standards. Differentiated preprocessing is performed on the validated multimodal data based on the data type; The differential preprocessing includes: Structured data is preprocessed using Z-score standardization and segmented into sliding window segments according to the grain quality evolution cycle; The sensor data is subjected to wavelet signal decomposition and noise reduction, followed by normalization. A pre-defined target detection model was used to segment the grain pile region using visual data, and color histograms, grain pile texture features, and rectangular features were extracted to obtain images of the grain pile region. A standardized multimodal dataset is generated through feature concatenation and standardization.

[0009] By adopting the above technical solutions, the heterogeneity problem of multimodal data fusion is solved. The target detection model is preset to the YOLOv8 model. Structured data is standardized by Z-score to eliminate dimensional differences. Combined with the sliding window segmentation to capture the evolution cycle of grain quality (such as the 30-day cycle), the periodicity of temporal features is improved. Sensor data is processed by wavelet denoising and normalization to suppress environmental noise interference, which is conducive to enhancing the sensitivity of dynamic features such as the frequency of pest activity and the rate of moisture migration. Visual data is accurately segmented into grain pile areas through the YOLOv8 model, and multi-dimensional spatial information such as color histogram and texture features is extracted, transforming visual information into feature vectors that can be quantified and analyzed.

[0010] In a preferred embodiment, this application includes the following: the spatiotemporal feature fusion analysis based on preprocessed multimodal data, the basic indicators of grain quality, and the rate of change of grain quality, comprising: Time series synchronization and alignment are performed on the preprocessed multimodal data; Based on a sliding window segmented according to the grain quality evolution cycle, a long short-term memory artificial neural network is used to extract temporal features, where the time step is equal to the window length. A pre-trained neural network is used to extract spatial features from images of the grain pile area, and combined with a temperature field heat map to generate spatial correlation features. The temperature field heat map is generated based on temperature data of the entire grain storage area. A multi-head attention mechanism is used to weight and fuse temporal and spatial features to generate a fused feature vector.

[0011] By adopting the above technical solution, the long short-term memory artificial neural network is an LSTM network; the pre-trained neural network is a ResNet-50 network; the LSTM network extracts temporal features (time step = grain quality evolution cycle), and the ResNet-50 network extracts spatial features of grain pile images. The temperature field heat map is introduced to generate spatial correlation features, realizing multi-scale alignment of temporal and spatial features. By weighted fusion of temporal features (such as temperature changes over 7 consecutive days) and spatial features (such as local hot spots in grain piles), the model can simultaneously capture the dynamic process of grain quality changes (such as pest spread trajectories) and spatial distribution patterns (such as clustering in humid and hot areas).

[0012] In a preferred embodiment, this application describes the generation of a grain quality change trend prediction model using a preset grain quality evolution law and the grain quality prediction feature space, comprising: Using the preset grain quality evolution law, key quality influencing factors are screened through mutual information entropy to obtain a feature importance matrix; Based on the key influencing factors after screening, the t-SNE algorithm is used to perform nonlinear dimensionality reduction on the feature matrix to obtain a quality evolution law map containing the main components of grain quality evolution. Obtain the critical point of quality mutation in the quality evolution pattern map; Design a two-stream spatiotemporal fusion neural network prediction model with an 8-head self-attention mechanism and a spatial attention module; Based on the aforementioned feature importance matrix and quality mutation critical point, and combined with a sliding window online update strategy, the parameters of the dual-stream spatiotemporal fusion neural network prediction model are optimized, ultimately outputting a grain quality change trend prediction model.

[0013] By adopting the above technical solution, mutual information entropy (MI) is used to screen key quality influencing factors (such as temperature and pest density), and t-SNE nonlinear dimensionality reduction is combined to generate a quality evolution law map, so as to eliminate redundant features. The evolution law of grain quality deterioration path is visualized through the quality evolution law map. The core feature subset (the top k high contribution factors) helps to reduce the number of prediction model parameters and model calculation, and improve the prediction inference speed of the grain quality change trend prediction model.

[0014] In a preferred embodiment of this application: the step of filtering key quality influencing factors through mutual information entropy to obtain a feature importance matrix includes: Define the quality influencing factor pool: F={T, H, CO2, O2, P, M, C, I}, where T is the grain storage temperature; H is the grain storage humidity; CO2 is the CO2 concentration; O2 is the O2 concentration; P is the pest density; M is the moisture content; C is the fatty acid value; and I is the reducing sugar content. Calculate the historical correlation between each quality influencing factor and grain quality grade: in, ∈F; Q is the grain quality grade label; Quality influencing factors whose absolute correlation value is greater than the set screening threshold are entered into the mutual information entropy screening stage. The characteristic matrix of quality evolution law is constructed as follows: , Where n is the number of samples. Let be the multimodal feature vector of the i-th sample; Calculate the mutual information value between each candidate quality impact factor and the quality label: Where q is the specific value of the grain quality grade label; Candidate quality impact factor The probability that the quality label Q takes a specified value simultaneously; Candidate quality impact factor The marginal probability distribution; The marginal probability distribution of the quality label Q; Sort the mutual information values ​​in descending order, and select the top k candidate quality influence factors whose cumulative contribution is greater than the preset contribution threshold to form the core feature subset. ; Generate a two-dimensional feature importance matrix: Where M is the feature dimension and n is the number of samples. This represents the contribution weight of the j-th feature of the i-th sample to the quality evolution.

[0015] By adopting the above technical solution, a dual-stream network architecture (temporal stream and spatial stream) is designed, consisting of an 8-head self-attention mechanism and a spatial attention module. The temporal stream refers to capturing the periodic patterns of grain quality changes (such as monthly fluctuations) through 8 parallel attention heads; the spatial stream refers to using the spatial attention module to focus on hot spots in the grain pile (such as high-temperature and high-humidity areas). At the same time, an online update strategy combined with a sliding window (7-day window + 0.95 forgetting factor) is used to dynamically adjust the model parameters, so that the model can maintain a prediction accuracy of more than 85% even under scenarios such as grain variety change and sudden changes in environmental parameters.

[0016] In a preferred embodiment of this application: the online update strategy for the sliding window sets the sliding window size to 7 days and the forgetting factor λ to 0.95; the method further includes: Obtain the actual quality change data corresponding to the current grain warehouse environmental parameters; The difference between the actual quality change data and the predicted grain quality change trend is analyzed to obtain the prediction error value. Based on the prediction error value, adjust the attention weights of the dual-stream spatiotemporal fusion neural network prediction model: If the prediction error value is greater than the preset error threshold, the model parameters are backtracked and updated, and the feature importance matrix is ​​recalculated. If the prediction error value is less than the preset error threshold, maintain the current model parameters and record the error distribution pattern; The forgetting factor is adjusted by statistically analyzing the error distribution using a sliding window.

[0017] By adopting the above technical solutions, a sliding window online update strategy (window size 7 days) and an error feedback mechanism are used. The error status of the model is statistically analyzed based on the error distribution rules, and the model deviation is adjusted in a timely manner to avoid the impact of accumulated errors on the model's prediction accuracy. At the same time, the forgetting factor can balance the weights of new and old data, avoid short-term noise from interfering with long-term trend prediction, and improve the long-term stability of the grain quality change trend prediction model.

[0018] In a preferred embodiment of this application: the temporal and spatial features are weighted and fused using a multi-head attention mechanism, wherein the formula for calculating the attention weights is as follows: in, For the first Attention weights of each temporal feature vector at the target time step t; For the first One time-series feature vector; The feature vector of the target time step t; The cosine similarity function; For the first One time-series feature vector; The output length of the time-series feature sequence; When outputting the fused feature vector, the temporal features are given a higher weight than the spatial features.

[0019] By adopting the above technical solution, in practical applications, temporal features can be assigned 60% weight and spatial features can be assigned 40% weight to emphasize and strengthen the temporal dependence of grain quality changes (such as the influence of temperature change rate on mold). This application uses a cosine similarity weighted attention mechanism to dynamically fuse temporal features and spatial features, and improves the detection sensitivity of local anomalies inside the grain warehouse by focusing on spatial features.

[0020] Secondly, the objective of this invention is achieved through the following technical solution: A multimodal data fusion system for predicting grain quality change trends in grain warehouses, the system comprising: The data acquisition and processing module is used to acquire multimodal data from the grain warehouse and perform preprocessing. The quality index analysis module is used to determine the basic quality indicators and the rate of change of grain quality based on preprocessed multimodal data. The feature fusion analysis module is used to perform spatiotemporal feature fusion analysis based on preprocessed multimodal data, the basic indicators of grain quality, and the rate of change of grain quality, and to construct a grain quality prediction feature space. The prediction model generation module generates a grain quality change trend prediction model by using a preset grain quality evolution law and the grain quality prediction feature space; the preset grain quality evolution law is obtained based on historical multimodal data and historical grain warehouse environmental parameters. The prediction execution module obtains the current grain warehouse environmental parameters and inputs the current grain warehouse environmental parameters into the grain quality change trend prediction model to output the grain quality change trend prediction result.

[0021] By adopting the above technical solutions, using multimodal data fusion, spatiotemporal feature alignment and high-precision prediction models, a leapfrog improvement has been achieved in grain quality prediction, from single-indicator judgment to multi-dimensional collaborative analysis, and from static models to dynamic adaptive systems.

[0022] Thirdly, the objective of this invention is achieved through the following technical solution: A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for predicting the trend of grain quality changes in a grain warehouse using multimodal data fusion.

[0023] Fourthly, the objective of this invention is achieved through the following technical solution: A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of a multimodal data fusion method for predicting grain quality change trends within a grain warehouse, as described above.

[0024] In summary, this application includes at least one of the following beneficial technical effects: 1. Based on the integrated analysis of multimodal data, a spatiotemporal feature fusion model is constructed to achieve comprehensive collaborative analysis of grain quality change trends. This application breaks through the information limitations of a single data source. Through the linkage analysis of basic grain quality indicators and grain quality change rates, it significantly improves the prediction accuracy of local anomalies (such as localized mold) and global trends (such as overall aging) in grain piles; meets the needs of intelligent warehouse management, and improves the accuracy and practicality of grain quality change trend prediction. Attached Figure Description

[0025] Figure 1 This is a flowchart of a method for predicting the trend of grain quality changes in a granary by multimodal data fusion in one embodiment of this application. Detailed Implementation

[0026] The present application will be further described in detail below with reference to the accompanying drawings.

[0027] In one embodiment, such as Figure 1 As shown, this application discloses a method for predicting the trend of grain quality changes in granaries by multimodal data fusion, which specifically includes the following steps: S1: Acquire multimodal data from the grain warehouse and perform preprocessing.

[0028] In this embodiment, since the grain storage environment has the characteristics of multi-parameter coupling and nonlinear change, a single data source is difficult to fully reflect the grain quality status. This embodiment achieves comprehensive perception and trend prediction of the grain storage environment and grain pile status through multi-modal data fusion.

[0029] Specifically, step S1 includes: S11: Based on the grain storage equipment deployment information, determine the multimodal data collection list.

[0030] In this embodiment, the multimodal data includes structured data, sensor data, and visual data; the grain silo equipment deployment information includes sensor location, sensor model, camera location, and data acquisition frequency, wherein the camera is mounted on a track robot laid within the grain silo, and the track robot moves back and forth on the track laid within the grain silo to acquire comprehensive high-definition images of the grain pile surface.

[0031] Structured data includes grain silo temperature (T), grain silo humidity (H), CO2 concentration, and O2 concentration, collected hourly; sensor data includes grain pile weight change rate (detected by moisture migration via pressure sensors) and pest activity video (a 10-second video segment is collected every 2 hours). Visual data consists of high-resolution images of the grain pile surface, captured daily from 9:00 to 10:00 by an orbiting robot.

[0032] S12: Based on the preset data quality assessment standards, verify the validity of the collected raw multimodal data.

[0033] In this embodiment, the validity verification of multimodal data includes: for structured data, checking data integrity and rationality, requiring missing data values ​​≤5%, and the rationality requirements for structured data, for example, the numerical range of temperature data is 0-40℃, and the humidity range is 10%-80%. After filtering out high-frequency noise through wavelet transform (db4 wavelet basis, decomposition level = 3), the sensor data verification signal strength must be ≥-85dBm; for visual data, the SSIM index is used to calculate the image sharpness of high-definition images of the grain pile surface, requiring that the obtained structural similarity ≥0.8 is a relatively clear image frame, which is retained for subsequent analysis, and the structural similarity is less than 0.8 is a blurry frame and needs to be discarded.

[0034] S13: Perform differential preprocessing on the validated multimodal data according to the data type; In this embodiment, the differential preprocessing includes: Structured data was preprocessed using Z-score standardization and segmented into sliding window segments according to the grain quality evolution cycle. Specifically, the Z-score standardization formula is: z = (structured data - historical mean) / historical standard deviation. The window length for the sliding window segment is 7 days, which corresponds to the grain quality evolution cycle. The sliding step size is 1 day, generating continuous time series segments through sliding window segmentation. Sensor data was decomposed into wavelet signals for denoising and then normalized, where normalization maps the sensor signal amplitude to the [0, 1] interval. Visual data was segmented into grain pile regions using a pre-defined target detection model, extracting color histograms, grain pile texture features, and rectangular features to obtain grain pile region images. The pre-defined target detection model is the YOLOv8 model (confidence threshold = 0.9) to segment the grain pile regions. Based on the segmented grain pile regions, color histograms (HSV space, bins = 8), GLCM texture features (contrast, correlation), and Hu rectangular features (regularity score ≥ 0.85) were calculated. A standardized multimodal dataset is generated through feature concatenation and standardization.

[0035] S2: Based on the preprocessed multimodal data, determine the basic indicators of grain quality and the rate of change of grain quality.

[0036] In this embodiment, the basic indicators of grain quality include fatty acid value (C), reducing sugar content (I) and microbial activity (implicit indicator), and the rate of change of grain quality includes the rate of temperature fluctuation and the rate of insect pest reproduction.

[0037] The calculation of basic indicators for grain quality includes: Fatty acid value (C) was calculated based on GB / T 5510-2017 standard by inversion using near-infrared spectroscopy (wavelength range 900-1700nm); reducing sugar content (I) was determined by anthrone colorimetric method, using a spectrophotometer at a wavelength of 620nm; microbial activity (implicit indicator) indirectly reflected the respiration intensity of the grain pile through the rate of change of CO2 concentration (ΔCO2 / Δt).

[0038] The formula for calculating the rate of temperature fluctuation in the rate of change of grain quality is: Where the time interval Δt = 1 hour; T(t) is the grain warehouse temperature at the current time (t); T(t−Δt) is the grain warehouse temperature at the previous time (t−Δt), representing the historical temperature status; the pest reproduction rate in the grain quality change rate is based on the Logistic growth model. The parameter r (reproduction rate) is fitted by the change in insect population density between video frames. Here, K is the carrying capacity, expressed as heads / cubic meter (heads / m³), representing the maximum number of pests the grain storage environment can support; t0 is the initial time, indicating the initial outbreak time of the pest, characterizing the critical time for pests to transition from the latent period to the active period; P(t) is the number of pests at time t, expressed as heads / cubic meter (heads / m³); and r is the intrinsic growth rate, expressed as 1 / day (d). -1 ( ), indicating the reproduction rate of pests under ideal conditions; E is a natural constant, approximately 2.718.

[0039] Specifically, step S2 includes: S21: Perform time series synchronization and alignment on the preprocessed multimodal data.

[0040] In this embodiment, the time reference of the preprocessed multimodal data is first unified, for example, by using a UTC timestamp; the sensor data (such as insect pest videos) is supplemented with minute-level time granularity using linear interpolation; and the visual data (such as daily grain pile images) is aligned to the hour level using time nearest neighbor matching. Then, time windows are divided according to the grain quality evolution cycle (7 days) to generate a synchronous data cube, and the window sliding step size is set to 1 day to generate continuous time series segments.

[0041] S22: Based on a sliding window segmented according to the grain quality evolution cycle, a long short-term memory artificial neural network is used to extract temporal features, where the time step is equal to the window length.

[0042] In this embodiment, the Long Short-Term Memory (LSTM) artificial neural network is used. The network architecture design includes: input dimensions including 8-dimensional temporal features (T / H / CO2 / O2 / P / M / C / I); the number of hidden nodes is 128; the time step is 7 days (the evolution cycle of grain quality); and the output dimension is 64 (compressed temporal feature vector). Feature compression reduces the dimensionality of the LSTM output to 64 dimensions through a fully connected layer, and the BatchNorm and LeakyReLU activation functions are applied.

[0043] S23: A pre-trained neural network is used to extract spatial features from the grain pile area image, and combined with the temperature field heat map to generate spatial correlation features. The temperature field heat map is generated based on the temperature data of the entire grain storage area.

[0044] In this embodiment, the pre-trained neural network is a ResNet-50 network. The image features of the grain pile area are extracted by using the pre-trained ResNet-50 network (with the weights of the first 15 layers frozen), and a 2048-dimensional global feature vector is output. The temperature field heat map is a 2D temperature field heat map generated based on multi-point temperature sensor data of various sub-areas of the grain warehouse and using the inverse distance weighted (IDW) interpolation algorithm. The heat map resolution is 0.5m × 0.5m, and the temperature range is 0-40℃.

[0045] Specifically, the temperature field thermogram is concatenated with visual features and input into a 3-layer convolutional network (using 3×3 convolutional kernels); spatial correlation features are extracted. in, To obtain a spatial correlation feature map after weighted summation, the skill retains local details and reflects the global distribution pattern; Let the coordinates be the points; This is a spatial location index variable, representing discrete location points sampled on or inside the grain pile. The total number of spatial locations represents the number of sampling points participating in feature fusion; coordinate point The spatial weights in the spatial association features are calculated by normalizing using the Softmax function; Coordinates of the grain pile surface or interior Temperature measurement value at the location; Visual features are image feature vectors. S24: The temporal and spatial features are weighted and fused through a multi-head attention mechanism to generate a fused feature vector.

[0046] In this embodiment, the multi-head attention mechanism is an 8-head attention mechanism with a head dimension of 32, used for dynamically weighting temporal or spatial features; the temporal features (64 dimensions) and spatial features (512 dimensions) are mapped to the same dimension (256 dimensions) through a fully connected layer; first, the cosine similarity between the temporal features and spatial features is calculated: in, Spatial features are static features extracted through CNN / pre-trained visual models, such as the color histogram of a grain pile image or local anomaly regions in a temperature field heatmap.

[0047] Generate fusion weights using Softmax: ,in, The temporal-spatial similarity score is used. Spatial-temporal similarity score; vector It includes two similarity scores.

[0048] Then, a weighted fusion calculation is performed to generate the final fused feature vector: in, For time-series feature weights; For spatial feature weights. If Greater than The model assigns higher weights to temporal features, such as scenarios where high temperatures accelerate mold growth; if Less than The model focuses on spatial features, such as localized pest outbreaks; temporal features capture the continuity of grain quality evolution (such as the increase of fatty acid values ​​over time); and spatial features capture local anomalies in grain piles (such as microbial aggregation in hot and humid areas).

[0049] For example, time series features Grain temperature rose from 25℃ to 30℃ over the past 7 days (upward trend); spatial characteristics A high-temperature hotspot (35℃) was found in the lower right corner of the grain pile; similarity calculation. They exhibit high correlation; weight allocation ; Fusion features The model predicts a "medium" quality level and triggers a local ventilation command in the lower right corner.

[0050] Furthermore, in another embodiment, the formula for calculating attention in the multi-head attention mechanism is: in, For the first Attention weights of each temporal feature vector at the target time step t; For the first One time-series feature vector; The feature vector of the target time step t; The cosine similarity function; For the first One time-series feature vector; The output length of the time-series feature sequence.

[0051] When outputting the fused feature vector, the temporal features are given a higher weight than the spatial features.

[0052] With the default values, the weight of time series features accounts for 60%, and the weight of spatial features accounts for 40%.

[0053] S3: Based on preprocessed multimodal data, basic grain quality indicators, and grain quality change rate, spatiotemporal feature fusion analysis is performed to construct a grain quality prediction feature space.

[0054] In this embodiment, due to the spatiotemporal heterogeneity of grain quality changes, to improve the accuracy of trend prediction for grain quality changes, this application first aligns structured data, sensor data, and visual data to a minute-level time granularity through timestamp matching; then, it uses an LSTM network (hidden layer nodes = 128, time step = 7 days) to extract temporal features, using the following formula: in, The current hidden state is used to encode historical information up to time step t (such as grain warehouse temperature fluctuations and pest activity patterns). The hidden state at time step t-1 carries historical time series information, such as the temperature trend of the grain pile over the past 7 days. This is the Sigmoid function, with an output range of (0, 1). The current input represents the multimodal input data at time step t. and Let be the weight matrix, where To connect to the hidden state To the current hidden state The weights are 128×128 (assuming the number of hidden layer nodes is 128). Connection input The weights to the previous hidden state have a dimension of 128×N (N is the dimension of the input features). The bias parameter for the hidden state has a dimension of 128×1 and is used to adjust the initial activation level of the hidden state.

[0055] S4: Generate a grain quality change trend prediction model by utilizing the preset grain quality evolution law and grain quality prediction feature space; the preset grain quality evolution law is obtained based on historical multimodal data and historical grain warehouse environmental parameters.

[0056] In this embodiment, step S4 includes: S41: Using the pre-defined laws governing the evolution of grain quality, key quality influencing factors are screened through mutual information entropy to obtain a feature importance matrix.

[0057] In this embodiment, the quality influencing factor pool is defined as: F={T, H, CO2, O2, P, M, C, I}, where T is the grain storage temperature; H is the grain storage humidity; CO2 is the CO2 concentration; O2 is the O2 concentration; P is the pest density; M is the moisture content; C is the fatty acid value; and I is the reducing sugar content. Calculate the historical correlation between each quality influencing factor and grain quality grade: in, ∈F; Q represents the grain quality grade label, specifically including Excellent, Good, Medium, and Poor. The grain quality grade label mainly references national standards (taking rice as an example, such as GB / T 20569-2006 "Rules for Judging the Storage Quality of Rice") and industry specifications, and is dynamically adjusted based on the evolution of grain quality (such as the increase of fatty acid value over time). Key indicators include physicochemical and biological indicators. Physicochemical indicators include fatty acid value, moisture content, and reducing sugar content; biological indicators include pest density and mold spore count.

[0058] For example, taking rice as an example, the "Excellent" grade label on grain quality indicates that the grain quality is at its best, and all indicators meet the national long-term storage standards. Specific characteristics include: fatty acid value ≤15 mg / 100g (for rice); moisture content ≤ safe storage moisture (e.g., ≤14.5% for japonica rice); no signs of heating, mold, or pests; normal odor; and fresh color. The "Good" grade on the grain quality label indicates good grain quality, but some indicators are starting to deviate from the ideal range. For example, the fatty acid value is 15-20 mg / 100g. At this time, the storage strategy is to maintain normal storage and monitor regularly. The moisture content is close to the safe upper limit, with slight local heating or pest infestation (insect density ≤5 heads / kg), and slight color changes. The storage strategy at this time is to start local ventilation, cooling, or pest control measures. The "Medium" grade on the grain quality label indicates that the grain quality has deteriorated significantly and intervention measures are required. Characteristics include a fatty acid value of 20-30 mg / 100g, excessive moisture content (e.g., japonica rice >15%), significant heating (local temperature ≥30℃), and pest infestation (insect density 5-20 heads / kg). The storage strategy at this time is forced ventilation, controlled atmosphere storage (e.g., nitrogen filling), or local fumigation. The "Poor" grade on the grain quality label indicates severely deteriorated grain quality, losing its edible or economic value. The characteristics are "fatty acid value ≥30 mg / 100g". If the grain has a moisture content of mg / 100g that is severely excessive (e.g., japonica rice >17%), widespread heating (temperature ≥35℃), insect outbreak (insect density >20 insects / kg), obvious mold, clumping, or rotten odor, or slight mold spots or rancid smell, the storage strategy should be to remove the grain from storage immediately and prohibit further storage.

[0059] Quality influencing factors whose absolute correlation value is greater than the set screening threshold are entered into the mutual information entropy screening stage; the screening threshold is set to 0.5.

[0060] The characteristic matrix of quality evolution law is constructed as follows: , Where n is the number of samples. Let be the multimodal feature vector of the i-th sample; Calculate the mutual information value between each candidate quality impact factor and the quality label: Where q is the specific value of the grain quality grade label; Candidate quality impact factor The probability that the quality label Q takes a specified value simultaneously; Candidate quality impact factor The marginal probability distribution; The marginal probability distribution of the quality label Q is defined; the mutual information values ​​are sorted in descending order, and the top k candidate quality influence factors with cumulative contribution greater than a preset contribution threshold are selected to form the core feature subset. The preset contribution threshold is 85%, and k takes the value of 5 in this embodiment, that is, the top 5 candidate quality influence factors constitute the core features.

[0061] Generate a two-dimensional feature importance matrix: Where M is the feature dimension and n is the number of samples. This represents the contribution weight of the j-th feature of the i-th sample to the quality evolution (normalized to [0, 1]).

[0062] S42: Based on the key influencing factors after screening, the t-SNE algorithm is used to perform nonlinear dimensionality reduction on the feature matrix to obtain a quality evolution law map containing the main components of grain quality evolution.

[0063] In this embodiment, the t-SNE algorithm (t-Distributed Stochastic Neighbor Embedding) is a nonlinear algorithm for dimensionality reduction and visualization of high-dimensional data. The parameters of the t-SNE algorithm are configured as follows: perplexity = 30 (matching the complexity of grain quality evolution), learning rate = 200 (balancing convergence speed and accuracy), and number of iterations = 1000 (ensuring convergence).

[0064] Specifically, the dimensionality reduction process involves reducing the 5-dimensional feature matrix (corresponding to the 5 core influencing factors selected above) to a 3-dimensional feature matrix, generating a quality evolution pattern map containing 3 major components of grain quality evolution, where the variance contribution rate of the first major component of grain quality evolution is ≥65% (retaining the major evolution component). The path of grain quality deterioration is displayed through a 3D scatter plot.

[0065] For example, five core influencing factors were selected using the mutual information entropy method: T (temperature): Correlation with fatty acid value ρ = 0.82; H (humidity): Correlation with mold spore count ρ = 0.75; P (insect density): Correlation with the proportion of insect-eaten particles ρ=0.68; C (fatty acid value): directly reflects the degree of oxidative degradation; I (reducing sugar content): associated with Maillard reaction products.

[0066] The generated feature importance matrix (partial example) is as follows: M= The matrix values ​​represent the contribution weights of each feature to the quality evolution.

[0067] Dimensionality reduction using the t-SNE algorithm: The original 5-dimensional features are reduced to a 3-dimensional embedding space. For example: First principal component (PC1): explained 68% of the variance (dominantly indicating the trend of quality deterioration); Second principal component (PC2): explains 22% of the variance (reflects the synergistic effect of temperature and humidity); Third principal component (PC3): explains 10% of variance (related to pests and oxidation reactions).

[0068] In the quality evolution pattern map, fatty acid values, temperature and humidity co-variation values, and pest density are selected to construct a 3D dot map. Color is used to map fatty acid values, for example, from blue to red, to indicate fatty acid values ​​from low to high; shape is used to mark storage stages, for example, from circles to rhombuses, to represent the early to late stages of storage; the size of the dots represents the pest density, with larger dots indicating higher pest density.

[0069] For example, the following key features can be observed from the map (using a staged evolutionary path as an example): Initial stage (excellent): Data points are concentrated in the lower left corner (low temperature, low humidity, low PC1 value), and fatty acid value <15 mg / 100g.

[0070] Interim (Good): Data points spread to the upper right (temperature fluctuations, increased humidity), PC1 value increases (15-20).

[0071] Late stage (medium / poor): Data points rise rapidly along the PC1 axis (fatty acid value > 25 mg / 100g), accompanied by the clustering of red areas (hot spots in high temperature and high humidity).

[0072] S43: Obtain the critical point of quality mutation in the quality evolution pattern map.

[0073] In this embodiment, the DBSCAN clustering algorithm is used to determine the critical point of quality mutation. The neighborhood radius of the DBSCAN clustering algorithm is ϵ=0.5 (determined based on the feature space density, with a minimum sample size minPts=5). Outliers are marked as critical points of quality mutation (e.g., the fatty acid value mutation threshold C=20 mg / 100g). The initially determined critical points of quality mutation are verified by combining domain knowledge verification, such as the national standard GB / T 20569-2006 which stipulates that when the fatty acid value is ≥30, the grain quality is "poor".

[0074] S44: Design a two-stream spatiotemporal fusion neural network prediction model with an 8-head self-attention mechanism and a spatial attention module.

[0075] In this embodiment, the network architecture of the dual-stream spatiotemporal fusion neural network prediction model includes a temporal stream and a spatial stream. The temporal stream is configured as a 4-layer Transformer encoder (8 heads, 128 hidden layers), with the input being temporal features within a sliding window (length = 7 days). The spatial stream is configured as a 3-layer convolutional network (3×3 kernels, stride = 1), with the input being an image of the grain pile area (resolution = 224×224×3). The fusion layer of the dual-stream spatiotemporal fusion neural network prediction model uses a gating mechanism (such as the sigmoid activation function) to dynamically fuse the features from both streams, and the output layer uses a Softmax + threshold segmentation composite structure, where the threshold value is 0.5.

[0076] S45: Based on the feature importance matrix and the critical point of quality mutation, combined with the sliding window online update strategy, the parameters of the dual-stream spatiotemporal fusion neural network prediction model are optimized, and the final output is a prediction model of grain quality change trend.

[0077] In this embodiment, the sliding window online update strategy sets the sliding window size to 7 days and the forgetting factor to 0.95. A backtracking update is triggered when the prediction error (MAE) > 5%.

[0078] Specifically, the dual-stream spatiotemporal fusion neural network prediction model employs adversarial training to enhance robustness. The generator simulates environmental parameter perturbations (following a normal distribution N(μ, 0.1μ)), and the discriminator uses a Wasserstein GAN (gradient penalty coefficient = 10). The model's parameters are optimized online by combining a feature importance recalculation mechanism. Feature importance recalculation refers to recalculating the feature importance matrix M every 10 windows and dynamically adjusting the model's input weights.

[0079] S5: Obtain the current grain storage environment parameters, input the current grain storage environment parameters into the grain quality change trend prediction model to output the grain quality change trend prediction results.

[0080] In this embodiment, the current grain storage environment parameters include temperature, humidity, gas composition, and pest activity. The gas composition is collected by an electrochemical sensor array with two channels (CO2 / O2), and the pest activity is collected by an infrared thermal imager. The output layer of the grain quality change trend prediction model uses a Softmax classifier (four categories: "excellent / good / medium / poor") to predict the grain quality grade. An LSTM decoder is used to generate prediction curves for fatty acid value, temperature fluctuation, and humidity change for the next 72 hours. For example, the prediction range of key indicators is as follows: fatty acid value: 12.5-15.8 mg / 100g, temperature fluctuation: 22-28℃.

[0081] Specifically, the input feature weights are dynamically adjusted based on the feature importance matrix. For example, if the current pest density suddenly increases (e.g., P increases by 30%), the weight coefficients of features P and I are increased.

[0082] Furthermore, based on the predicted trend of grain quality changes, graded control instructions are generated. For example, if the predicted fatty acid value is ≥25 mg / 100g, the quality grade is "poor". At this time, the treatment measures of "emergency release + harmless treatment" can be adopted. If the predicted daily temperature difference is ≥5℃, the quality grade is "good". At this time, the treatment measures of activating the backup fan can be adopted.

[0083] In one embodiment, a method for predicting the trend of grain quality changes in a grain warehouse based on multimodal data fusion further includes: S451: Obtain the actual quality change data corresponding to the current grain warehouse environmental parameters.

[0084] In this embodiment, grain pile samples are collected weekly to test the fatty acid value, moisture content, and mold spore count of the grain. The testing cycle is 7 days / time, synchronized with the sliding window cycle.

[0085] S452: Perform a difference analysis between the actual quality change data and the predicted results of grain quality change trends to obtain the prediction error value.

[0086] In this embodiment, the formula for calculating the prediction error value is as follows: ,in This represents the total number of samples extracted in real time. Index for sample variables; For quality indicators (such as fatty acid values) predicted by the model; These are laboratory measured values.

[0087] The calculated prediction error values ​​are classified into categories. For example, when E > 10%, it is a severe error; when 4% < E < 10%, it is a moderate error; and when E ≤ 4%, it is an acceptable error.

[0088] S453: Adjust the attention weights of the dual-stream spatiotemporal fusion neural network prediction model based on the prediction error value.

[0089] In this embodiment, the adjustment of attention weights includes error backpropagation and spatial weight correction. Error backpropagation adjusts the query, key, and value matrix weights of the multi-head attention mechanism using gradient descent. The specific adjustment values ​​can be customized based on the knowledge and experience of experts in the relevant field, with a focus on time steps with larger errors (such as the 7-day window with the largest prediction deviation). Spatial weight correction refers to locally correcting the fusion weights of the temperature field heatmap and visual features. For example, if the prediction error of a certain region is too high, the weight of that region in spatial attention is increased.

[0090] S454: If the prediction error value is greater than the preset error threshold, trigger the backtracking update of model parameters and recalculate the feature importance matrix.

[0091] In this embodiment, the model parameter backtracking update is based on a preset error threshold. A backtracking update is triggered immediately when there is a severe error, and generally triggered when there are three accumulated errors. The update process is as follows: 1) Recalculate the feature importance matrix (S41); 2) Freeze the pre-trained ResNet-50 weights and fine-tune the parameters of the fully connected layers; 3) Update model parameters using the Adam optimizer (learning rate = 0.001).

[0092] S455: If the prediction error value is less than the preset error threshold, maintain the current model parameters and record the error distribution pattern.

[0093] In this embodiment, the error distribution pattern can be statistically analyzed using the mean, variance, and bias of the error. Meanwhile, if the error is ≤4% for 10 consecutive windows, the weights of the LSTM network are frozen, and fine-tuning is performed only through the attention mechanism and shallow fully connected layers to ensure the operational stability of the grain quality change trend prediction model.

[0094] S456: Adjust the forgetting factor by statistically analyzing the error distribution using a sliding window.

[0095] In this embodiment, the default value of the forgetting factor is 0.95. Using a 7-day sliding window, the standard deviation and mean within the sliding window are calculated. When the standard deviation within the sliding window is <0.4, the forgetting factor remains at 0.95; otherwise, it is adjusted according to the formula: "Adjusted forgetting factor = 0.95 - 0.02 (standard deviation - 0.05)". This application also addresses the sources of attention weight heatmap positioning errors (such as sensor drift in a certain area leading to increased error).

[0096] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0097] In one embodiment, a multimodal data fusion system for predicting the trend of grain quality changes in a granary is provided, which corresponds to the multimodal data fusion method for predicting the trend of grain quality changes in a granary described in the above embodiment.

[0098] A multimodal data fusion system for predicting grain quality change trends in grain warehouses includes a data acquisition and processing module, a quality index analysis module, a feature fusion analysis module, a prediction model generation module, and a prediction execution module. Detailed descriptions of each functional module are as follows: The data acquisition and processing module is used to acquire multimodal data from the grain warehouse and perform preprocessing. The quality index analysis module is used to determine the basic quality indicators and the rate of change of grain quality based on preprocessed multimodal data. The feature fusion analysis module is used to perform spatiotemporal feature fusion analysis based on preprocessed multimodal data, basic grain quality indicators, and grain quality change rate, and to construct a grain quality prediction feature space. The prediction model generation module uses the preset grain quality evolution law and grain quality prediction feature space to generate a grain quality change trend prediction model; the preset grain quality evolution law is obtained based on historical multimodal data and historical grain warehouse environmental parameters. The prediction execution module obtains the current grain storage environment parameters and inputs them into the grain quality change trend prediction model to output the grain quality change trend prediction results.

[0099] For specific limitations regarding the multimodal data fusion-based grain quality change trend prediction system in granaries, please refer to the limitations of the multimodal data fusion-based grain quality change trend prediction method in granaries mentioned above, which will not be repeated here. Each module in the aforementioned multimodal data fusion-based grain quality change trend prediction system in granaries can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0100] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: S1: Acquire multimodal data from the grain warehouse and perform preprocessing; S2: Based on the preprocessed multimodal data, determine the basic indicators of grain quality and the rate of change of grain quality; S3: Based on preprocessed multimodal data, basic grain quality indicators, and grain quality change rate, spatiotemporal feature fusion analysis is performed to construct a grain quality prediction feature space; S4: Generate a grain quality change trend prediction model by utilizing the preset grain quality evolution law and grain quality prediction feature space; the preset grain quality evolution law is obtained based on historical multimodal data and historical grain storage environmental parameters. S5: Obtain the current grain storage environment parameters, input the current grain storage environment parameters into the grain quality change trend prediction model to output the grain quality change trend prediction results.

[0101] Each module in the system can be implemented entirely or partially through software, hardware, or a combination thereof; the above modules can be embedded in the processor of the computer device in hardware form or independent of it, or they can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0102] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: S1: Acquire multimodal data from the grain warehouse and perform preprocessing; S2: Based on the preprocessed multimodal data, determine the basic indicators of grain quality and the rate of change of grain quality; S3: Based on preprocessed multimodal data, basic grain quality indicators, and grain quality change rate, spatiotemporal feature fusion analysis is performed to construct a grain quality prediction feature space; S4: Generate a grain quality change trend prediction model by utilizing the preset grain quality evolution law and grain quality prediction feature space; the preset grain quality evolution law is obtained based on historical multimodal data and historical grain storage environmental parameters. S5: Obtain the current grain storage environment parameters, input the current grain storage environment parameters into the grain quality change trend prediction model to output the grain quality change trend prediction results.

[0103] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0104] In one embodiment, particularly according to an embodiment of the invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, embodiments of the invention include a computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication module, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the various functions defined in this invention.

[0105] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0106] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for predicting the trend of grain quality changes in granaries using multimodal data fusion, characterized in that, include: Acquire multimodal data from within the grain warehouse and perform preprocessing; Based on the preprocessed multimodal data, determine the basic indicators of grain quality and the rate of change of grain quality; Based on the preprocessed multimodal data, the basic indicators of grain quality, and the rate of change of grain quality, a spatiotemporal feature fusion analysis is performed to construct a grain quality prediction feature space. A grain quality change trend prediction model is generated by using the preset grain quality evolution law and the grain quality prediction feature space; the preset grain quality evolution law is obtained based on historical multimodal data and historical grain storage environmental parameters. Obtain the current grain storage environment parameters, and input the current grain storage environment parameters into the grain quality change trend prediction model to output the grain quality change trend prediction result.

2. The method for predicting the trend of grain quality changes in granaries by multimodal data fusion according to claim 1, characterized in that, The multimodal data includes structured data, sensor data, and visual data; The acquisition and preprocessing of multimodal data within the grain warehouse includes: Based on the grain depot equipment deployment information, a multimodal data collection list was determined; The validity of the collected raw multimodal data is verified based on the preset data quality assessment standards. Differentiated preprocessing is performed on the validated multimodal data based on the data type; The differential preprocessing includes: Structured data is preprocessed using Z-score standardization and segmented into sliding window segments according to the grain quality evolution cycle; The sensor data is subjected to wavelet signal decomposition and noise reduction, followed by normalization. A pre-defined target detection model was used to segment the grain pile region using visual data, and color histograms, grain pile texture features, and rectangular features were extracted to obtain images of the grain pile region. A standardized multimodal dataset is generated through feature concatenation and standardization.

3. The method for predicting the trend of grain quality changes in granaries by multimodal data fusion according to claim 2, characterized in that, The spatiotemporal feature fusion analysis based on preprocessed multimodal data, the basic indicators of grain quality, and the rate of change of grain quality includes: Time series synchronization and alignment are performed on the preprocessed multimodal data; Based on a sliding window segmented according to the grain quality evolution cycle, a long short-term memory artificial neural network is used to extract temporal features, where the time step is equal to the window length. A pre-trained neural network is used to extract spatial features from images of the grain pile area, and combined with a temperature field heat map to generate spatial correlation features. The temperature field heat map is generated based on temperature data of the entire grain storage area. A multi-head attention mechanism is used to weight and fuse temporal and spatial features to generate a fused feature vector.

4. The method for predicting the trend of grain quality change in granaries by multimodal data fusion according to claim 1, characterized in that, The process of generating a grain quality change trend prediction model by utilizing the preset grain quality evolution law and the grain quality prediction feature space includes: Using the preset grain quality evolution law, key quality influencing factors are screened through mutual information entropy to obtain a feature importance matrix; Based on the key influencing factors after screening, the t-SNE algorithm is used to perform nonlinear dimensionality reduction on the feature matrix to obtain a quality evolution law map containing the main components of grain quality evolution. Obtain the critical point of quality mutation in the quality evolution pattern map; Design a two-stream spatiotemporal fusion neural network prediction model with an 8-head self-attention mechanism and a spatial attention module; Based on the aforementioned feature importance matrix and quality mutation critical point, and combined with a sliding window online update strategy, the parameters of the dual-stream spatiotemporal fusion neural network prediction model are optimized, ultimately outputting a grain quality change trend prediction model.

5. The method for predicting the trend of grain quality changes in granaries by multimodal data fusion according to claim 4, characterized in that, The process of filtering key quality influencing factors through mutual information entropy to obtain a feature importance matrix includes: Define the quality influencing factor pool: F={T, H, CO2, O2, P, M, C, I}, where T is the grain storage temperature; H is the grain storage humidity; CO2 is the CO2 concentration; O2 is the O2 concentration; P is the pest density; M is the moisture content; C is the fatty acid value; and I is the reducing sugar content. Calculate the historical correlation between each quality influencing factor and grain quality grade: in, ∈F; Q is the grain quality grade label; Quality influencing factors whose absolute correlation value is greater than the set screening threshold are entered into the mutual information entropy screening stage. The characteristic matrix of quality evolution law is constructed as follows: , Where n is the number of samples. Let be the multimodal feature vector of the i-th sample; Calculate the mutual information value between each candidate quality impact factor and the quality label: Where q is the specific value of the grain quality grade label; Candidate quality impact factor The probability that the quality label Q takes a specified value simultaneously; Candidate quality impact factor The marginal probability distribution; The marginal probability distribution of the quality label Q; Sort the mutual information values ​​in descending order, and select the top k candidate quality influence factors whose cumulative contribution is greater than the preset contribution threshold to form the core feature subset. ; Generate a two-dimensional feature importance matrix: Where M is the feature dimension and n is the number of samples. This represents the contribution weight of the j-th feature of the i-th sample to the quality evolution.

6. The method for predicting the trend of grain quality change in a granary based on multimodal data fusion according to claim 5, characterized in that, The online update strategy for the sliding window sets the sliding window size to 7 days and the forgetting factor to 0.95; the method further includes: Obtain the actual quality change data corresponding to the current grain warehouse environmental parameters; The difference between the actual quality change data and the predicted grain quality change trend is analyzed to obtain the prediction error value. Based on the prediction error value, adjust the attention weights of the dual-stream spatiotemporal fusion neural network prediction model: If the prediction error value is greater than the preset error threshold, the model parameters are backtracked and updated, and the feature importance matrix is ​​recalculated. If the prediction error value is less than the preset error threshold, maintain the current model parameters and record the error distribution pattern; The forgetting factor is adjusted by statistically analyzing the error distribution using a sliding window.

7. A method for predicting the trend of grain quality changes in a granary based on multimodal data fusion according to claim 4 or 6, characterized in that, The process involves weighted fusion of temporal and spatial features using a multi-head attention mechanism, where the calculation formula for the attention weights is as follows: in, For the first Attention weights of each temporal feature vector at the target time step t; For the first One time-series feature vector; The feature vector of the target time step t; The cosine similarity function; For the first One time-series feature vector; The output length of the time-series feature sequence; When outputting the fused feature vector, the temporal features are given a higher weight than the spatial features.

8. A multimodal data fusion system for predicting the trend of grain quality changes in grain warehouses, characterized in that, The system includes: The data acquisition and processing module is used to acquire multimodal data from the grain warehouse and perform preprocessing. The quality index analysis module is used to determine the basic quality indicators and the rate of change of grain quality based on preprocessed multimodal data. The feature fusion analysis module is used to perform spatiotemporal feature fusion analysis based on preprocessed multimodal data, the basic indicators of grain quality, and the rate of change of grain quality, and to construct a grain quality prediction feature space. The prediction model generation module generates a grain quality change trend prediction model by using a preset grain quality evolution law and the grain quality prediction feature space; the preset grain quality evolution law is obtained based on historical multimodal data and historical grain warehouse environmental parameters. The prediction execution module obtains the current grain warehouse environmental parameters and inputs the current grain warehouse environmental parameters into the grain quality change trend prediction model to output the grain quality change trend prediction result.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for predicting the trend of grain quality change in a granary by multimodal data fusion as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method for predicting the trend of grain quality change in a grain warehouse by multimodal data fusion as described in any one of claims 1 to 7.

Images