Multi-modal fusion food stock and shelf life collaborative analysis system

Abstract

The application relates to the technical field of food storage management, and discloses a multi-modal fusion food storage and shelf life collaborative analysis system, which comprises the following modules: a multi-modal data acquisition module for acquiring image recognition, environmental sensing, inventory turnover log and temperature and humidity regulation record data; a storage state analysis module for extracting warehouse space features based on image recognition data; a shelf life prediction module for performing time series analysis on the inventory turnover log to generate a prediction vector; a collaborative analysis module for performing time-space collaborative modeling on the environmental sensing and temperature and humidity regulation record to generate a correlation distribution map; and a decision output module for fusing information of the modules to generate analysis conclusions and regulation instructions. The system can also generate an evaluation report according to feedback, iteratively optimize the mechanism, realize accurate collaborative analysis and dynamic regulation of food storage and shelf life, and improve the efficiency of warehouse management and the quality and safety of food.

Description

Technical Field

[0001] The present invention relates to the technical field of food warehousing management, and particularly to a multi-modal fusion collaborative analysis system for food storage quantity and shelf life. Background Art

[0002] In modern food supply chain management, the food warehousing link faces the dual challenges of accurate monitoring of storage quantity and effective management of shelf life. Most traditional food warehousing management systems adopt a single-modal data collection and analysis method, which is difficult to comprehensively and accurately reflect the complex conditions of the warehousing environment.

[0003] From the perspective of storage quantity management, traditional methods usually rely on manual inventory or simple sensor counting, and cannot obtain key information such as stack density distribution, container overflow degree, and cargo space occupancy rate of the warehousing space in real time and dynamically. This makes it difficult for warehousing managers to accurately grasp the actual storage status of the inventory, and it is easy to出现 inventory backlog or shortage, affecting the efficient operation of the supply chain.

[0004] In terms of shelf life management, traditional systems often only make simple calculations based on the food's warehousing time and nominal shelf life, ignoring the actual impact of warehousing environmental factors (such as temperature and humidity fluctuations) and inventory turnover patterns on the food's shelf life. For example, abnormal fluctuations in temperature and humidity may accelerate food spoilage, and unreasonable inventory turnover rates may lead to long-term backlogs of some foods, increasing the risk of approaching expiration. In addition, traditional systems lack in-depth time series analysis of inventory turnover logs, cannot accurately identify key features such as batch warehousing intervals and turnover deviation degrees, and are difficult to achieve accurate prediction of the food's shelf life.

[0005] At the same time, traditional systems have obvious deficiencies in data fusion and collaborative analysis. Different types of data (such as image recognition data, environmental sensing data, inventory turnover logs, etc.) are often stored and analyzed independently, lacking an effective spatio-temporal collaborative modeling mechanism, unable to establish the correlation between storage quantity and shelf life, and difficult to achieve comprehensive and collaborative management of food warehousing.

[0006] Moreover, the decision-making support ability of traditional systems is limited, lacking a dynamic update mechanism, unable to adjust analysis conclusions and control instructions in a timely manner according to real-time data, resulting in the lag of management strategies and being difficult to adapt to the complex and changeable warehousing environment.

[0007] With the rapid development of the food industry and the continuous improvement of consumers' requirements for food safety, there is an urgent need for a system that can fuse multi-modal data and achieve collaborative analysis of food storage quantity and shelf life, so as to improve the accuracy and efficiency of warehousing management, and reduce food loss and safety risks. Summary of the Invention

[0008] The purpose of this invention is to provide a multimodal fusion-based collaborative analysis system for food storage and shelf life, in order to solve the problems mentioned in the background art.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a multimodal fusion-based collaborative analysis system for food storage and shelf life, the system comprising:

[0010] Multimodal data acquisition module: used to collect multimodal data sets in real time in food storage scenarios, including image recognition data, environmental sensor data, inventory turnover logs, and temperature and humidity control records;

[0011] Storage status analysis module: Based on the image recognition data, it extracts storage space features through a multi-source data alignment algorithm. The features include stacking density distribution, container fullness, and storage space occupancy rate.

[0012] Shelf life prediction module: Performs time-series pattern analysis on the inventory turnover log, and generates a shelf life prediction vector using a time-series feature fusion algorithm. The prediction vector includes batch entry interval, turnover rate deviation and near-expiration risk coefficient.

[0013] Collaborative analysis module: performs spatiotemporal collaborative modeling on the environmental sensor data and temperature and humidity control records to generate a distribution map of the correlation between storage capacity and shelf life; the spatiotemporal collaborative modeling includes gridding of the storage area and clustering of temperature and humidity fluctuations;

[0014] Decision output module: By dynamically updating the storage space characteristics, shelf life prediction vector and correlation distribution map, it generates multi-dimensional collaborative analysis conclusions and outputs a sequence of control instructions.

[0015] Preferably, the implementation steps of the multi-source data alignment algorithm include:

[0016] The image recognition data is calibrated for viewing angle deviation to separate two-dimensional contour information from three-dimensional spatial coordinates;

[0017] Based on the distribution threshold of stacking density using a sliding window, the proportion of standard storage locations and the area of ​​abnormal stacking are calculated.

[0018] A dynamic benchmark is generated based on the time gradient rate of change of container fullness, and multi-dimensional warehouse space characteristics are constructed by combining the distribution threshold.

[0019] Preferably, the time-series feature fusion algorithm includes:

[0020] The inventory turnover log is sliced ​​into time windows, and the batch turnover rate and the proportion of near-expiry products in each window are extracted.

[0021] An abnormal interval in the warehousing interval is detected by using a time series decomposition algorithm, and a preliminary prediction label is generated by combining it with the near-expiration risk coefficient.

[0022] The turnover rate deviation is dynamically weighted by fuzzy logic rules, and a standardized shelf life prediction vector is output.

[0023] Preferably, the spatiotemporal collaborative modeling includes:

[0024] Based on the temperature and humidity monitoring points of environmental sensor data, the warehouse grid units are divided, and the temperature fluctuation value and humidity uniformity within each unit are statistically analyzed.

[0025] A spatiotemporal correlation matrix is ​​constructed based on the control command sequence recorded by temperature and humidity control to identify the spatiotemporal distribution pattern of the control delay region;

[0026] By fusing temperature fluctuation values ​​and control delay patterns using an association mapping algorithm, a distribution map of the correlation between reserves and shelf life is generated.

[0027] Preferably, the implementation steps of the dynamic update mechanism include:

[0028] Sliding window difference calculation is performed on the characteristics of the storage space to extract the trend of storage changes;

[0029] The feature weights are dynamically adjusted based on the time decay factor of the shelf life prediction vector, combined with the spatial weight coefficients of the correlation distribution map.

[0030] Tensor fusion of reserve change trends and weighted features is performed to generate incremental analysis feature sequences.

[0031] Incremental feature sequences are superimposed onto the historical analysis model through an online update mechanism, outputting real-time collaborative analysis conclusions.

[0032] Preferably, the optimization method for the viewpoint deviation calibration includes: calculating the viewpoint deviation threshold and spatial resolution range based on the distribution of historical image data, iteratively correcting the alignment error between the two-dimensional contour and the three-dimensional coordinates through a bilinear interpolation algorithm, adjusting the statistical accuracy of the distribution threshold in conjunction with a dynamic benchmark, and optimizing the stability of warehouse space feature extraction.

[0033] Preferably, the parameter configuration method for the fuzzy logic rules includes: defining the membership function of the turnover rate deviation and the influence factor of the risk coefficient based on historical turnover data, optimizing the confidence threshold and weight allocation of the fuzzy rule base through particle swarm optimization algorithm, and dynamically updating the membership function parameters based on real-time data stream.

[0034] Preferably, the method for constructing the association mapping algorithm includes:

[0035] Define the normalized weighting coefficients for temperature fluctuation values ​​and control delay patterns;

[0036] Based on the standardized correlation benchmark value of the area of ​​the storage grid unit, the weighted result is mapped to the preset correlation interval through a piecewise linear function;

[0037] The standardized correlation values ​​are associated with the time window index to form a spatiotemporal correlation distribution matrix.

[0038] Preferably, the online update mechanism includes:

[0039] Version identifiers are generated based on the timestamps of incremental analysis feature sequences, and the difference detection algorithm is used to identify the regions of difference between the current features and the historical models.

[0040] The feature weights of the different regions are dynamically superimposed according to the time decay factor to update the calculation logic of the collaborative analysis conclusion.

[0041] Preferably, the system further includes:

[0042] Based on the feedback results of the control command sequence, a collaborative control effect evaluation report is generated;

[0043] Based on the feature weight allocation strategy of the dynamic update mechanism of the evaluation report iteration optimization, a closed-loop collaborative optimization mechanism is formed.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] The multimodal fusion-based collaborative analysis system for food storage and shelf life provided by this invention collects multi-source data in real time, including image recognition data, environmental sensor data, inventory turnover logs, and temperature and humidity control records, through a multimodal data acquisition module. This provides comprehensive and rich data support for the system's analysis. The storage status analysis module, based on image recognition data, uses a multi-source data alignment algorithm to extract storage space characteristics such as stacking density distribution, container fullness, and storage location occupancy rate. This enables precise perception of the storage space status, allowing managers to monitor the storage status of inventory in real time and providing a basis for rational storage space planning.

[0046] The shelf-life prediction module performs time-series pattern analysis on inventory turnover logs and uses a time-series feature fusion algorithm to generate a shelf-life prediction vector that includes batch entry interval, turnover rate deviation, and near-expiry risk coefficient. This fully considers the impact of inventory turnover patterns on food shelf-life, improves the accuracy of shelf-life prediction, and helps to identify near-expiry risky foods in advance and take corresponding measures.

[0047] The collaborative analysis module performs spatiotemporal collaborative modeling of environmental sensor data and temperature and humidity control records. Through operations such as gridding of storage areas and clustering of temperature and humidity fluctuations, it generates a distribution map of the relationship between storage quantity and shelf life, establishing the correlation between storage quantity and shelf life. This allows managers to intuitively understand the mutual influence between food storage quantity and shelf life in different storage areas, providing support for the formulation of scientific storage management strategies.

[0048] The decision output module integrates warehouse space characteristics, shelf-life prediction vectors, and correlation distribution maps through a dynamic update mechanism to generate multi-dimensional collaborative analysis conclusions and output a sequence of control instructions. This dynamic update mechanism can continuously optimize the analysis model based on real-time data, improving the accuracy and timeliness of decision-making and enabling dynamic control of the warehouse environment and inventory management.

[0049] Furthermore, the system includes generating a collaborative control effect evaluation report based on the feedback results of the control command sequence, and iteratively optimizing the feature weight allocation strategy of the dynamic update mechanism based on the evaluation report, forming a closed-loop collaborative optimization mechanism. This enables the system to continuously learn and optimize during actual operation, further improving system performance and management effectiveness, reducing food waste, improving warehouse management efficiency, and ensuring food quality and safety. Attached Figure Description

[0050] Figure 1 This is a schematic diagram illustrating the working principle of the multimodal fusion-based collaborative analysis system for food storage and shelf life described in this invention.

[0051] Figure 2 Design diagram of the temporal feature fusion algorithm;

[0052] Figure 3 Design drawings for spatiotemporal collaborative modeling;

[0053] Figure 4 Design diagram for the implementation steps of the dynamic update mechanism;

[0054] Figure 5 A design diagram for constructing a method for the association mapping algorithm. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Please see Figures 1-5The present invention relates to a multimodal fusion collaborative analysis system for food storage and shelf life. The system includes a multimodal data acquisition module, which is capable of real-time acquisition of a multimodal data set from a food storage scenario. This set includes image recognition data, environmental sensor data, inventory flow logs, and temperature and humidity control records. Specific implementation details are as follows:

[0057] The storage status analysis module extracts warehouse space features based on image recognition data using a multi-source data alignment algorithm. These features include stacking density distribution, container fullness, and storage location occupancy rate. The shelf-life prediction module performs time-series pattern analysis on inventory turnover logs and uses a time-series feature fusion algorithm to generate a shelf-life prediction vector. This vector includes batch entry interval, turnover rate deviation, and near-expiration risk coefficient. The collaborative analysis module performs spatiotemporal collaborative modeling of environmental sensor data and temperature and humidity control records to generate a storage-shelf-life correlation distribution map. The spatiotemporal collaborative modeling includes warehouse area gridding and temperature and humidity fluctuation clustering. The decision output module integrates warehouse space features, shelf-life prediction vectors, and correlation distribution maps through a dynamic update mechanism to generate multi-dimensional collaborative analysis conclusions and output a sequence of control instructions.

[0058] Example 1: This example details the implementation of the multi-source data alignment algorithm. This algorithm is applied to the storage status analysis module, extracting storage space features based on image recognition data. The specific steps are as follows:

[0059] Viewpoint deviation calibration is performed on image recognition data. Due to the varying installation positions and angles of cameras in warehouse scenarios, the acquired images may exhibit viewpoint deviations, affecting the accuracy of subsequent feature extraction. In this process, the viewpoint deviation threshold and spatial resolution range are first calculated based on the distribution of historical image data. Historical image data covers images from different time periods and warehouse areas; statistical analysis of this data determines common viewpoint deviation ranges and corresponding spatial resolutions. Then, a bilinear interpolation algorithm is used to iteratively correct the alignment error between the two-dimensional contour information and the three-dimensional spatial coordinates. The bilinear interpolation algorithm calculates approximate values ​​for unknown points based on the grayscale or coordinate values ​​of adjacent pixels, gradually reducing the deviation between the two-dimensional contour and the three-dimensional coordinates through multiple iterations. During the iteration process, the statistical accuracy of the distribution threshold is adjusted in conjunction with a dynamic benchmark. The dynamic benchmark is dynamically determined based on the actual situation of the current image and the characteristics of historical data; it adapts to image changes in different scenarios, making the statistical accuracy of the distribution threshold more accurate, thereby optimizing the stability of warehouse spatial feature extraction.

[0060] A distribution threshold for stacking density is calculated based on a sliding window. The size and step size of the sliding window are set according to the actual size of the storage space and the image resolution. The window slides across the image, analyzing the stacking situation within each window and calculating the stacking density. By statistically analyzing the stacking density across multiple windows, the distribution of stacking density is obtained, and a distribution threshold is determined. This threshold is used to distinguish between normal and abnormal stacking areas. During the statistical process, the impact of different food types and packaging methods on stacking density needs to be considered to ensure the rationality of the distribution threshold. Based on the obtained distribution threshold, the proportion of standard storage locations and the area of ​​abnormal stacking are calculated. The proportion of standard storage locations reflects the normal utilization of the storage space, while the area of ​​abnormal stacking indicates the size of the area with stacking problems. These two indicators are crucial for evaluating the utilization efficiency of storage space.

[0061] A dynamic benchmark is generated based on the time gradient rate of change of container fullness. Container fullness is an indicator of the amount of food stored in a container, and its time gradient rate of change reflects how quickly the container fullness changes over time. By analyzing the time gradient changes of container fullness in historical data, the rules for generating the dynamic benchmark are determined. The dynamic benchmark can automatically adjust over time and according to changes in storage conditions to adapt to different storage scenarios. Combined with previously obtained distribution thresholds, multi-dimensional storage space features are constructed. These multi-dimensional storage space features include information from multiple dimensions such as stacking density distribution, container fullness, and storage location occupancy rate. These features describe the state of the storage space from different perspectives, providing comprehensive data support for subsequent storage analysis and decision-making.

[0062] Throughout the implementation of the multi-source data alignment algorithm, each step is interconnected and influences the others. The accuracy of viewpoint deviation calibration directly affects the calculation accuracy of stacking density and container overflow, while sliding window statistics and dynamic benchmark generation provide the foundation for constructing accurate warehouse space characteristics. By continuously optimizing the parameters and algorithms of each step, the multi-source data alignment algorithm can efficiently and accurately extract warehouse space characteristics, laying a solid foundation for the normal operation of the entire food storage and shelf-life collaborative analysis system.

[0063] Example 2: This example details the implementation of the time-series feature fusion algorithm, which is used in the shelf-life prediction module to perform time-series pattern analysis on inventory turnover logs and generate shelf-life prediction vectors. The specific operations are as follows:

[0064] The inventory turnover log is sliced ​​into time windows. The inventory turnover log records information such as the time and quantity of food batches' inbound and outbound operations. Based on the actual needs of warehouse management and the characteristics of the data, the size of the time window is determined; for example, it can be divided into time units such as hours, days, or weeks. The inventory turnover log is divided into multiple consecutive time windows in chronological order, with each window containing turnover data for a specific time period. Then, for the log data within each window, batch turnover rate and the proportion of near-expiry products are extracted. The batch turnover rate is determined by calculating the ratio of the number of food batches outbound to the number inbound within that window; it reflects the speed of food turnover during that time period. The proportion of near-expiry products is the ratio of the quantity of near-expiry food within that window to the total quantity of food, used to measure the risk level of near-expiry food during that time period. During the extraction process, the data needs to be cleaned and preprocessed to remove outliers and erroneous data to ensure the accuracy and reliability of the extracted indicators.

[0065] Anomaly intervals in the warehousing interval are detected using a time series decomposition algorithm. This algorithm breaks down warehousing interval data into trend, seasonal, and random components. Analysis of these components identifies intervals that deviate significantly from the normal pattern—these are considered anomaly intervals. The warehousing interval refers to the time interval between two adjacent batches of food entering the warehouse. Abnormal warehousing intervals can impact food inventory management and shelf-life prediction. When detecting anomaly intervals, reasonable decomposition algorithm parameters and anomaly judgment thresholds need to be set based on the characteristics of historical warehousing data and the actual situation of warehousing operations. Preliminary prediction labels are generated by combining the near-expiration risk coefficient. The near-expiration risk coefficient is an indicator determined by comprehensively considering factors such as the remaining shelf life of the food and the storage environment, used to assess the near-expiration risk faced by the food. By combining the detected anomaly intervals with the near-expiration risk coefficient, preliminary shelf-life prediction labels are generated for each time window. These labels provide a preliminary assessment of the shelf-life of the food within that window.

[0066] The turnover rate deviation is dynamically weighted using fuzzy logic rules. Turnover rate deviation refers to the degree of deviation between the current batch turnover rate and the historical average turnover rate, reflecting abnormal turnover conditions. Based on fuzzy set theory, the fuzzy logic rules divide the turnover rate deviation into different fuzzy categories, such as "low deviation," "medium deviation," and "high deviation," and assign corresponding weights to each category. In practical applications, the membership function and the influence factor of the risk coefficient are defined based on historical turnover data. The membership function describes the degree to which the turnover rate deviation belongs to a certain fuzzy category, while the influence factor adjusts the strength of the risk coefficient's effect on the weighted result. The confidence threshold and weight allocation of the fuzzy rule base are optimized using a particle swarm optimization (PSO) algorithm. PSO is an optimization algorithm that simulates the foraging behavior of bird flocks to find the optimal solution in the solution space, thereby optimizing the parameters of the fuzzy rule base and making the fuzzy logic rules more reasonable. Simultaneously, the membership function parameters are dynamically updated based on real-time data streams to adapt to constantly changing warehouse turnover conditions. Finally, based on the dynamically weighted results, a standardized shelf-life prediction vector is output. This vector contains information such as batch warehousing interval, turnover rate deviation, and near-expiration risk coefficient, providing a quantitative basis for subsequent shelf-life analysis and decision-making.

[0067] In the implementation of the time-series feature fusion algorithm, each step is closely interconnected. The rationality of the time window slicing directly affects the accuracy of subsequent indicator extraction, the precision of the time series decomposition algorithm determines the effectiveness of anomaly interval detection, and the optimization degree of the fuzzy logic rules relates to the reliability of the final prediction vector. Through careful design and continuous optimization of each step, the time-series feature fusion algorithm is ensured to accurately extract useful time-series features from inventory turnover logs and generate a scientifically reasonable shelf-life prediction vector.

[0068] Example 3: This example details the implementation of spatiotemporal collaborative modeling. This modeling is used in the collaborative analysis module to fuse environmental sensor data with temperature and humidity control records to generate a storage-shelf-life correlation distribution map. The specific process is as follows:

[0069] The warehouse is divided into grid units based on temperature and humidity monitoring points from environmental sensors. In food storage scenarios, temperature and humidity monitoring points are typically distributed within the storage space according to certain rules to collect real-time temperature and humidity data for each area. Based on the location and number of monitoring points, the entire storage space is divided into multiple grid units, with each grid unit corresponding to one or more monitoring points. The size and shape of the grid units can be adjusted according to factors such as the actual layout of the storage space, the distribution density of monitoring points, and the severity of temperature and humidity changes, ensuring that the temperature and humidity data within each grid unit are representative. After division, the temperature fluctuation value and humidity uniformity within each unit are statistically analyzed. The temperature fluctuation value is determined by calculating the difference between the maximum and minimum temperature values ​​within the unit over a period of time, reflecting the magnitude of temperature variation within that unit. Humidity uniformity measures the evenness of humidity distribution within the unit and can be assessed by calculating the standard deviation of humidity values ​​at each monitoring point.

[0070] A spatiotemporal correlation matrix is ​​constructed based on the control command sequence from temperature and humidity control records. The temperature and humidity control records detail the time, control equipment, and control parameters for temperature and humidity control in the storage environment, forming a control command sequence. The spatiotemporal correlation matrix describes the relationship between control commands and temperature and humidity changes in different areas of the storage space. Rows in the matrix represent different control commands, and columns represent different grid cells or time points. When constructing the matrix, it is necessary to analyze the execution time, impact range, and actual effect on the temperature and humidity of each control command, converting this information into element values ​​in the matrix. Analysis of the spatiotemporal correlation matrix allows for the identification of spatiotemporal distribution patterns in control delay areas. Control delay areas refer to those regions where temperature and humidity fail to reach the expected targets in a timely manner after the control commands are executed. Identifying the spatiotemporal distribution patterns of these areas is crucial for optimizing temperature and humidity control strategies.

[0071] A correlation mapping algorithm is used to fuse temperature fluctuation values ​​and regulation delay patterns to generate a distribution map of the correlation between storage capacity and shelf life. The algorithm is constructed as follows: First, normalized weighting coefficients for temperature fluctuation values ​​and regulation delay patterns are defined. These coefficients adjust the importance of temperature fluctuation values ​​and regulation delay patterns during the fusion process and can be determined based on the actual impact of temperature fluctuations and regulation delays on food shelf life in historical data. Next, the correlation benchmark value is standardized based on the area of ​​the storage grid cells. The benchmark value is a fundamental indicator for measuring the correlation between storage capacity and shelf life; standardizing it by grid cell area eliminates the impact of differences in cell area on the correlation analysis. Then, a piecewise linear function maps the weighted results to a preset correlation interval. This function maps the weighted results to corresponding correlation intervals based on different weighting ranges, making the correlation results more intuitive and easier to understand. Finally, the standardized correlation values ​​are correlated with a time window index to form a spatiotemporal correlation distribution matrix. The time window index records the time interval corresponding to each correlation value, ensuring the distribution matrix contains both spatial and temporal information.

[0072] Throughout the implementation of spatiotemporal collaborative modeling, the rational division of storage grid units is fundamental, ensuring the accuracy and representativeness of temperature and humidity data. The construction of the spatiotemporal correlation matrix is ​​crucial, revealing the intrinsic relationship between control commands and temperature and humidity changes. The optimization of the correlation mapping algorithm is the core, effectively integrating temperature fluctuation values ​​with control delay patterns to ultimately generate an accurate distribution map of storage capacity-shelf life correlation. This modeling process fully considers the spatiotemporal characteristics of the storage environment and the dynamic process of temperature and humidity control, providing comprehensive and accurate spatial correlation information for food storage management and shelf-life prediction. This helps storage managers better understand the shelf-life of food in different storage areas and at different times, thereby formulating more scientific and reasonable storage management strategies. By continuously optimizing the parameters and algorithms of each stage, the accuracy and efficiency of spatiotemporal collaborative modeling can be improved, making it better adaptable to complex and ever-changing food storage environments.

[0073] Example 4: This example details the implementation of the dynamic update mechanism, which is used in the decision output module to generate real-time collaborative analysis conclusions by fusing warehouse space characteristics, shelf-life prediction vectors, and correlation distribution maps. The specific steps are as follows:

[0074] A sliding window differential calculation is performed on the characteristics of the storage space. These characteristics include stacking density distribution, container fullness, and storage location occupancy rate, which change in real time with the inbound and outbound operations of food. The size of the sliding window is set according to the actual needs of warehousing operations and the frequency of characteristic changes; for example, it can be set to 1 hour, half a day, or a day. The sliding window moves along the time axis with a fixed step size, collecting storage space characteristic data within each window. Then, the difference between the characteristic data of the current window and the previous window is calculated to obtain the sliding window differential result. By analyzing the differential results, the trend of storage volume change is extracted, such as whether the storage volume is increasing, decreasing, or remaining stable, and the rate of change. For example, if the differential result of the storage location occupancy rate is consistently positive within a certain period, it indicates that the storage space occupancy is continuously increasing during that period, and the storage volume is showing an upward trend.

[0075] The shelf-life prediction vector is dynamically weighted based on a time decay factor. This vector contains information such as batch entry intervals, turnover rate deviations, and near-expiration risk coefficients. This information is highly time-sensitive, and its importance to the current shelf-life analysis gradually decreases over time. The time decay factor describes the degree of this time decay; it is a coefficient that gradually decreases over time. Smaller weights are assigned to shelf-life prediction vectors from earlier time points, while larger weights are assigned to vectors from more recent time points. Simultaneously, spatial weight coefficients from a correlation distribution map are used. The correlation distribution map reflects the correlation between storage volume and shelf life in different storage areas. Different spatial areas have different impacts on the overall analysis conclusions; therefore, corresponding spatial weight coefficients need to be set for each spatial area. For example, areas with larger temperature and humidity fluctuations have a more significant impact on food shelf life, and their spatial weight coefficients are relatively larger. By comprehensively considering the time decay factor and spatial weight coefficients, each feature is dynamically weighted, making the weighted features more reflective of the current situation.

[0076] Tensor fusion is used to combine the trend of storage changes with weighted features to generate an incremental analysis feature sequence. Tensor fusion is a method for comprehensively processing data of different dimensions and types, preserving multi-dimensional information and inherent relationships. The trend of storage changes reflects the changes in storage space characteristics over time, while weighted features are comprehensive features that integrate time and space factors. The tensor fusion algorithm integrates these two data sets to generate an incremental analysis feature sequence containing time, space, and feature dimensions. This sequence records the incremental feature information at each update, reflecting the dynamic changes in the system state. For example, when the stacking density in a certain area changes, the incremental analysis feature sequence will correspondingly record the temporal and spatial manifestations of this change.

[0077] An online update mechanism overlays incremental analysis feature sequences onto the historical analysis model, outputting real-time collaborative analysis conclusions. The online update mechanism operates as follows: A version identifier is generated based on the timestamps of the incremental analysis feature sequences. The timestamps precisely record the generation time of each incremental feature, and the version identifier identifies the model version at different points in time for management and traceability. A difference detection algorithm identifies the regions of difference between the current features and the historical model. This algorithm compares the incremental analysis feature sequences with the data in the historical analysis model, identifying the changed parts, i.e., the difference regions. For example, when the temperature and humidity control delay mode changes in a certain area, the difference detection algorithm can identify that area as a difference region. The feature weights of the difference regions are dynamically overlaid according to a time decay factor; that is, the feature weights of the difference regions are adjusted based on the time elapsed, with more weights added to more recent times and less added to weights of more distant times. In this way, the collaborative analysis conclusion calculation logic is updated, allowing the historical analysis model to continuously absorb new incremental feature information and gradually evolve into a model that better reflects the current situation. Finally, based on the updated model, real-time collaborative analysis conclusions are output, providing warehouse management personnel with the latest decision-making basis.

[0078] In the implementation of the dynamic update mechanism, each step revolves around real-time data processing and continuous model optimization. Sliding window differential calculation ensures timely capture of changing trends in warehouse space characteristics; dynamic weighting based on time decay factors and spatial weight coefficients makes feature fusion more reasonable; tensor fusion technology guarantees effective integration of multi-dimensional data; and the online update mechanism enables real-time model evolution. For example, in a food storage scenario, when a new batch of food enters the warehouse, the stacking density and location occupancy rate in the warehouse space characteristics will change. This changing trend can be detected in a timely manner through sliding window differential calculation. Simultaneously, the batch information of the newly entered food updates the shelf-life prediction vector, and its time decay factor determines the weight of this information in the current analysis. The spatial weight coefficient of the corresponding entry area in the correlation distribution map affects the overall analysis. After generating incremental feature sequences through tensor fusion, the online update mechanism overlays them onto the historical model, enabling the model to reflect the impact of this entry operation on storage capacity and shelf life in real time. Finally, it outputs collaborative analysis conclusions containing the latest information, guiding warehouse management personnel to make corresponding adjustments. This dynamic update mechanism can adapt to the constantly changing nature of data in the food storage environment, ensuring that the system always maintains accurate analysis and prediction of the current state, and providing strong technical support for the collaborative management of food storage and shelf life.

[0079] Example 5: This example details the implementation of the closed-loop collaborative optimization mechanism in the system. This mechanism optimizes the feature weight allocation strategy of the dynamic update mechanism by adjusting the feedback results of the control instructions. The specific process is as follows:

[0080] Once the decision output module generates a sequence of control instructions, the warehouse management system executes the corresponding control operations, such as adjusting temperature and humidity equipment parameters, optimizing storage location allocation, and triggering early warnings for near-expiry foods. At this time, the system collects feedback results from the control instruction sequence in real time, covering information from multiple dimensions. For example, after executing a temperature and humidity control instruction, the temperature and humidity monitoring values ​​in the environmental sensor data will change, requiring the collection of deviations between the actual temperature and humidity data of each grid unit and the control target values; after executing a storage location adjustment instruction, the stacking density distribution and storage location occupancy rate in the image recognition data will generate new states, requiring the recording of changes in the adjusted warehouse space characteristics; after executing a near-expiry food processing instruction, the inventory turnover log will update the relevant batch's outbound records or status markers, requiring the extraction of changes in indicators such as turnover rate and the proportion of near-expiry products.

[0081] The system generates a collaborative control effect evaluation report based on the collected feedback results. The construction of this evaluation report requires the integration and analysis of multi-source feedback data. Taking temperature and humidity control as an example, the actual temperature and humidity fluctuation values ​​in the environmental sensor data are compared with the target values ​​of the control commands. The control delay time and accuracy deviation of each grid unit are calculated, and it is analyzed whether the control delay area is consistent with the historical patterns in the spatiotemporal correlation matrix to determine the effectiveness of the control strategy. If the temperature in a certain area still fails to reach the target value within the expected time after control, it is necessary to analyze the cause of the delay—whether it is insufficient equipment power, obstructed ventilation paths, or other environmental factors—by combining the attributes of the warehouse grid units in that area (such as area and cargo stacking density) and the layout of the control equipment.

[0082] In the feedback evaluation of storage location adjustments, the adjusted stacking density distribution is extracted through image recognition data, the increase in the proportion of standard storage locations is calculated, and changes in the area of ​​abnormal stacking are observed. For example, after executing a storage location optimization instruction in a certain area, the container fullness returns from an over-threshold state to the normal range. The impact of this adjustment on the overall warehouse space utilization rate needs to be recorded. At the same time, combined with inventory turnover logs, it is necessary to check whether the inbound and outbound efficiency of food in that area has improved after the adjustment, and whether the turnover rate deviation has changed due to the storage location adjustment.

[0083] Feedback on instructions for handling near-expiry food products requires tracking the inventory turnover of the relevant batches. If a batch of food products is given priority for release after an alert is triggered, the actual release time of the batch needs to be compared with the remaining shelf life at the time of the alert. This analysis should examine whether the release operation was timely, and also review the impact of the batch's release on the overall turnover rate deviation and near-expiry risk coefficient, assessing the degree of matching between the alert mechanism and the control instructions.

[0084] After the evaluation report is generated, the system iteratively optimizes the feature weight allocation strategy of the dynamic update mechanism based on the report content. Specifically, if the evaluation finds a significant deviation between the spatiotemporal distribution pattern of the temperature and humidity control delay area and the historical correlation distribution map, it indicates that the normalized weight coefficients of the temperature fluctuation value and the control delay pattern set in the original spatiotemporal collaborative modeling may need to be adjusted. In this case, the system will recalculate the weight coefficients corresponding to different temperature and humidity fluctuation intervals based on the actual control effects of each region in the feedback data. For example, if the shelf life prediction deviation is large in a certain high-temperature and high-humidity region after control, it indicates that the weight of the temperature and humidity fluctuation on the shelf life in this region is underestimated. It is necessary to increase the weight coefficient of the temperature fluctuation value in the correlation mapping algorithm for this region, and at the same time adjust the spatial weight coefficient of this region in the dynamic update mechanism to give it a higher priority in subsequent analysis.

[0085] If the feedback from the storage location adjustment shows that the impact of changes in stacking density distribution on turnover rate deviation exceeds expectations, the system will re-examine the fuzzy logic rule parameters for turnover rate deviation in the time-series feature fusion algorithm. For example, if the turnover rate of a certain type of food decreases faster under high-density stacking conditions, the membership function of the turnover rate deviation for this type of food needs to be redefined using historical circulation data, and the confidence threshold optimized by the particle swarm optimization algorithm needs to be adjusted to make the fuzzy logic rules more aligned with actual business scenarios. Furthermore, the time decay factor of this feature will be adjusted in the dynamic update mechanism to make it more sensitive to recent changes in turnover rate.

[0086] In the feedback optimization of near-expiry food handling, if it is found that the near-expiry risk coefficient of a certain type of food fails to accurately reflect its spoilage rate in actual control, the system will recalibrate the calculation model of the near-expiry risk coefficient based on the actual data in the evaluation report. For example, if the near-expiry warning time for a certain dairy product in an area with large temperature and humidity fluctuations is much earlier than the actual spoilage time, it indicates that the environmental factors in that area have set the influence factor on shelf life too high. It is necessary to reduce the weight of environmental sensor data in the shelf life prediction vector, and at the same time adjust the weight allocation strategy of this feature in the dynamic update mechanism to make subsequent predictions closer to the actual situation.

[0087] The entire closed-loop collaborative optimization mechanism is implemented based on specific warehousing scenarios. For example, after executing temperature and humidity control commands, a large fresh food storage center discovered a significant delay in temperature control in a corner of the cold storage area, leading to a deviation in the predicted shelf life of fruits and vegetables in that area. By collecting temperature and humidity feedback data from that area and generating an evaluation report, the system identified that the delay was caused by poor ventilation in that corner. It then adjusted the spatial weight coefficient of that grid unit, adding a weight to the influence of temperature fluctuations in the dynamic update mechanism. Subsequently, when similar control commands are executed, the system will prioritize temperature and humidity changes in that area, adjusting the weight allocation of feature fusion to make the analysis conclusions more accurate. As another example, after adjusting the storage location of a batch of biscuits, the outbound efficiency improved. The system optimized the correlation weight between the stacking density and turnover rate of this type of food through feedback data, enabling the dynamic update mechanism to more accurately assess the impact of storage location adjustments on storage capacity and shelf life in subsequent analyses.

[0088] This closed-loop mechanism, through a continuous feedback-evaluation-optimization process, enables the system to adapt to changes in different food types, storage environments, and business processes. Each execution of a control command becomes training data for the optimization model. As feedback data accumulates, the feature weight allocation strategy of the dynamic update mechanism continuously evolves, ultimately achieving a coordinated improvement in the accuracy of inventory analysis and shelf-life prediction. In practical applications, this closed-loop optimization does not rely on preset experimental data or fixed parameters, but rather adapts entirely based on feedback from real-world storage scenarios, ensuring that the system maintains its accurate and coordinated analytical capabilities regarding food inventory and shelf-life throughout long-term operation.

[0089] Example 6: In modern family life, the demand for intelligent home bar food storage is increasing. To meet families' needs for efficient food management, proper storage, and ensuring food freshness, a food storage and shelf-life system has emerged. This system can be effectively applied to multiple intelligent home bar food storage scenarios, achieving precise and coordinated management of food storage and shelf-life.

[0090] The multimodal data acquisition module plays a fundamental supporting role in the entire system. It can collect multimodal data sets of food storage scenarios in real time, covering image recognition data, environmental sensor data, inventory flow logs, and temperature and humidity control records. In the multi-combination smart home bar, high-definition cameras are installed to collect image recognition data and monitor the placement and stacking of food in the bar in real time; at the same time, various environmental sensors, such as temperature and humidity sensors and light sensors, are deployed to obtain environmental sensor data; the bar's inventory management software records every food entry and exit operation, forming an inventory flow log; and the temperature and humidity control equipment records its operating status and control parameters. These data together constitute the multimodal data set.

[0091] The storage status analysis module extracts storage space features based on image recognition data using a multi-source data alignment algorithm. In the smart home bar, the image recognition data collected by the camera first undergoes viewpoint deviation calibration to separate two-dimensional contour information from three-dimensional spatial coordinates. For example, by analyzing the distribution of historical image data, the viewpoint deviation threshold and spatial resolution range are calculated, and the alignment error between the two-dimensional contour and the three-dimensional coordinates is iteratively corrected using a bilinear interpolation algorithm. Next, based on the distribution threshold of stacking density using a sliding window, the proportion of standard storage locations and the area of ​​abnormal stacking are calculated. For food containers of different shapes and sizes within the bar, a dynamic benchmark is generated based on the time gradient change rate of container overflow. Combined with the distribution threshold, multi-dimensional storage space features are constructed, including stacking density distribution, container overflow, and storage location occupancy rate. Through these features, the system can clearly understand the storage status of food within the bar, providing a basis for subsequent management.

[0092] The shelf-life prediction module performs time-series pattern analysis on inventory turnover logs and uses a time-series feature fusion algorithm to generate a shelf-life prediction vector. In the smart home bar, the inventory turnover log records information such as the entry and exit times of each batch of food. These logs are sliced ​​into time windows to extract the batch turnover rate and the proportion of near-expiry products within each window. Then, a time-series decomposition algorithm is used to detect abnormal intervals in the entry interval, and preliminary prediction labels are generated by combining this with the near-expiry risk coefficient. Next, the turnover rate deviation is dynamically weighted using fuzzy logic rules. Based on historical turnover data, the membership function of the turnover rate deviation and the influencing factors of the risk coefficient are defined. The confidence threshold and weight allocation of the fuzzy rule base are optimized using a particle swarm optimization algorithm, and the membership function parameters are dynamically updated based on real-time data streams. Finally, a standardized shelf-life prediction vector is output, which includes information such as batch entry interval, turnover rate deviation, and near-expiry risk coefficient. In this way, the system can predict the shelf-life of food in advance and remind users to dispose of near-expiry food in a timely manner.

[0093] The collaborative analysis module performs spatiotemporal collaborative modeling of environmental sensor data and temperature and humidity control records to generate a storage-shelf-life correlation distribution map. In the smart home bar, storage grid units are divided based on temperature and humidity monitoring points from environmental sensor data, and the temperature fluctuation and humidity uniformity within each unit are statistically analyzed. A spatiotemporal correlation matrix is ​​constructed based on the control command sequence from the temperature and humidity control records to identify the spatiotemporal distribution patterns of control delay areas. Through a correlation mapping algorithm, normalized weighting coefficients are defined for temperature fluctuation values ​​and control delay patterns. Based on the standardized correlation benchmark value of the storage grid unit area, the weighted results are mapped to a preset correlation interval using a piecewise linear function. The standardized correlation values ​​are then correlated with a time window index to form a spatiotemporal correlation distribution matrix, thereby generating the storage-shelf-life correlation distribution map. This distribution map allows the system to visually display the correlation between food storage and shelf-life in different areas of the bar.

[0094] The decision output module integrates storage space features, shelf-life prediction vectors, and correlation distribution maps through a dynamic update mechanism to generate multi-dimensional collaborative analysis conclusions and output a sequence of control instructions. In the smart home bar, sliding window difference calculations are performed on storage space features to extract storage change trends. Feature weights are dynamically adjusted based on the time decay factor of the shelf-life prediction vector, combined with the spatial weight coefficients of the correlation distribution map. The storage change trend and weighted features are fused using tensors to generate an incremental analysis feature sequence. A version identifier is generated based on the timestamp of the incremental analysis feature sequence. A difference detection algorithm identifies the regions of difference between the current features and the historical model, and the feature weights of the regions of difference are dynamically superimposed according to the time decay factor to update the collaborative analysis conclusion calculation logic. An online update mechanism superimposes the incremental analysis feature sequence onto the historical analysis model, outputting real-time collaborative analysis conclusions. For example, if the system analysis finds that a certain area has a high risk of food nearing its expiration date, and the temperature and humidity conditions in that area are unfavorable for food preservation, it will output control instructions, such as adjusting the temperature and humidity of that area or reminding users to prioritize consuming food nearing its expiration date.

[0095] The system also generates a collaborative control effect evaluation report based on the feedback results of the control command sequence. For example, after executing a control command to adjust temperature and humidity, the system monitors the temperature and humidity changes in the area and the trend of food shelf life changes to evaluate the control effect. Based on the feature weight allocation strategy of the iterative optimization dynamic update mechanism of the evaluation report, a closed-loop collaborative optimization mechanism is formed to continuously improve the accuracy and effectiveness of the system for multi-combination intelligent home bar food storage management.

[0096] Users can interact with the system via a mobile application. The system displays real-time information on the food storage status, shelf-life predictions, and control suggestions within the bar area. Users can also manage food storage and retrieval through the application, facilitating convenient and quick management of food storage in their home bar. In this way, the system provides an efficient and intelligent management solution for multi-unit smart home bar food storage, improving the quality and efficiency of home food management.

[0097] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0098] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multimodal fusion food inventory and shelf life collaborative analysis system, characterized in that, Comprise: Multimodal data acquisition module: for real-time acquisition of multimodal data set of food warehouse scene, the multimodal data set includes image recognition data, environmental sensing data, inventory turnover log and temperature and humidity control record; Reserve state analysis module: based on the image recognition data, the warehouse space features are extracted by multi-source data alignment algorithm, the features include pile density distribution, container overflow degree and storage occupancy rate; Shelf life prediction module: time series pattern analysis is performed on the inventory turnover log, time series feature fusion algorithm is adopted to generate shelf life prediction vector, the prediction vector includes batch warehousing interval, turnover rate deviation and near-expiry risk coefficient; Synergistic analysis module: time-space synergistic modeling is performed on the environmental sensing data and temperature and humidity control record to generate reserve-shelf life correlation distribution map; the time-space synergistic modeling includes warehouse area gridding and temperature and humidity fluctuation clustering; Decision output module: the warehouse space features, shelf life prediction vector and correlation distribution map are fused through a dynamic updating mechanism to generate multi-dimensional synergistic analysis conclusion and output control instruction sequence; The implementation steps of the dynamic updating mechanism include: The sliding window difference calculation is performed on the warehouse space features to extract the reserve change trend; The feature weight is dynamically adjusted based on the time decay factor of the shelf life prediction vector, and the spatial weight coefficient of the correlation distribution map is combined; The reserve change trend and weighted features are tensor fused to generate an incremental analysis feature sequence; The incremental analysis feature sequence is superimposed on the historical analysis model through an online updating mechanism to output real-time synergistic analysis conclusion.

2. The multimodal fused food stock and shelf life co-analytic system of claim 1, wherein, The implementation steps of the multi-source data alignment algorithm include: The image recognition data is subjected to perspective deviation calibration to separate two-dimensional contour information and three-dimensional space coordinates; The distribution threshold of the sliding window statistical pile density is calculated to obtain the standard storage occupancy ratio and abnormal pile area; The dynamic reference is generated according to the time gradient change rate of the container overflow degree, and the multi-dimensional warehouse space features are constructed in combination with the distribution threshold. 3.The multimodal fusion food stock and shelf life synergistic analysis system of claim 1, wherein, The time series feature fusion algorithm includes: The time window slicing is performed on the inventory turnover log to extract the batch turnover rate and near-expiry product proportion in each window; The time series decomposition algorithm is used to detect the abnormal interval of the warehousing interval, and the preliminary prediction label is generated in combination with the near-expiry risk coefficient; The turnover rate deviation is dynamically weighted through fuzzy logic rules to output the standardized shelf life prediction vector.

4. The multimodal fused food stock and shelf life co-analytic system of claim 1, wherein, The time-space synergistic modeling includes: The warehouse grid unit is divided according to the temperature and humidity monitoring points of the environmental sensing data, and the temperature fluctuation value and humidity uniformity in each unit are counted; The time-space correlation matrix is constructed based on the control instruction sequence of the temperature and humidity control record to identify the time-space distribution pattern of the control delay area; The reserve-shelf life correlation distribution map is generated by fusing the temperature fluctuation value and the control delay pattern through the correlation mapping algorithm.

5. The multimodal fused food stock and shelf life co-analytic system of claim 2, wherein, The optimization method of the perspective deviation calibration includes: calculating the perspective deviation threshold and the spatial resolution range according to the historical image data distribution, iteratively correcting the alignment error of the two-dimensional contour and the three-dimensional coordinates through the bilinear interpolation algorithm, adjusting the statistical precision of the distribution threshold in combination with the dynamic reference, and optimizing the stability of the warehouse space feature extraction.

6. The multimodal fused food stock and shelf life co-analytic system of claim 3, wherein, The parameter configuration method of the fuzzy logic rule comprises: defining a membership function of turnover rate deviation degree and an influence factor of risk coefficient according to historical flow data, optimizing a confidence threshold and weight distribution of the fuzzy rule base through a particle swarm algorithm, and dynamically updating parameters of the membership function based on real-time data flow.

7. The multimodal fused food stock and shelf life co-analytic system of claim 4, wherein, The construction method of the correlation mapping algorithm comprises: defining a normalized weight coefficient of temperature fluctuation value and regulation delay mode; mapping the weighted result to a preset correlation interval through a piecewise linear function according to the area standardized correlation reference value of the warehouse grid unit; associating the standardized correlation value with a time window index to form a distribution matrix of space-time correlation.

8. The multimodal fused food stock and shelf life co-analytic system of claim 1, wherein, The online updating mechanism comprises: generating a version identifier according to the timestamp of the incremental analysis feature sequence, identifying the difference area of the current feature and the historical model through a difference detection algorithm; dynamically superimposing the feature weight of the difference area according to a time decay factor to update the collaborative analysis conclusion calculation logic.

9. The multimodal fused food stock and shelf life co-analytic system of claim 1, wherein, Further comprising: generating a collaborative regulation effect evaluation report according to the feedback result of the regulation instruction sequence; iteratively optimizing the feature weight distribution strategy of the dynamic updating mechanism based on the evaluation report to form a closed-loop collaborative optimization mechanism.

Images