Cold chain warehouse system construction method based on multi-modal AI large model and cold chain warehouse system

Abstract

The invention discloses a cold chain warehouse system construction method based on a multi-modal AI large model and a cold chain warehouse system, and belongs to the field of intelligent cold chain warehousing, and the method comprises the steps: deploying various types of sensors in a target cold chain warehouse, carrying out the preprocessing of collected data through an edge gateway, and transmitting the processed data to a cloud end; the cloud end receives the data of the edge gateway, and stores the data to a multi-modal database of a target cold chain warehouse of a cloud computing platform deployed at the cloud end; processing data in the multi-modal database through a multi-modal fusion model of the cloud computing platform to obtain fusion modal features of different modals; and processing the fused feature tensor by using a Transform encoder to obtain general advanced features, and designing a multi-task output head according to the business requirements of the target cold chain warehouse to process the advanced features. The operation efficiency and the product safety level of the cold chain warehouse are improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent cold chain warehousing, and in particular to a cold chain warehouse system construction method and a cold chain warehouse system based on a multimodal AI large model. Background Art

[0002] Cold chain logistics, as a crucial component of modern logistics systems, plays a crucial role in areas such as food safety and pharmaceutical transportation. Smart spaces deploy a large number of heterogeneous sensors and intelligent devices to collect multimodal data in real time, providing a data foundation for intelligent cold chain logistics management. However, the heterogeneity of multimodal data, the real-time requirements, and device coordination challenges pose significant challenges to existing technologies. Traditional data processing methods struggle to effectively integrate dispersed, multi-source, and heterogeneous information, resulting in incomplete environmental perception, high decision-making latency, and difficulty adapting to the complex demands of dynamic scenarios. Furthermore, with the rapid development of the cold chain logistics industry, traditional cold chain logistics monitoring systems primarily rely on single-source temperature sensor data and lack the ability to comprehensively analyze multi-source data. This results in low warehouse demand forecasting accuracy and an inability to promptly identify potential demand risks. Furthermore, existing monitoring systems commonly suffer from insufficient data fusion and inadequate inter-modal correlation analysis, making it difficult to effectively leverage multimodal information to accurately predict cold chain warehouse demand trends. Furthermore, existing inventory management systems often rely on regular manual inspections and record-keeping, which is inefficient and prone to human error. The lack of effective data analysis methods makes it impossible to fully utilize real-time data for accurate prediction and optimized decision-making, making it difficult for warehouse systems to transform from passive response to active prediction and intelligent management. Summary of the Invention

[0003] One of the purposes of the present invention is to provide a method for constructing a cold chain warehouse system based on a multimodal AI large model to solve the problem in the existing technology that cold chain warehouses are difficult to effectively integrate scattered multi-source heterogeneous information and cannot fully utilize real-time data to achieve accurate prediction and optimization decision-making of cold chain warehouses.

[0004] The present invention is implemented through the following technical solution, a method for constructing a cold chain warehouse system based on a multimodal AI large model, comprising the following steps: deploying multiple types of sensors in a target cold chain warehouse, preprocessing the collected data with an edge gateway, and sending the processed data to the cloud; the cloud receives the data from the edge gateway and stores the data in a multimodal database of the target cold chain warehouse on a cloud computing platform deployed on the cloud; processing the data in the multimodal database through a multimodal fusion model of the cloud computing platform to obtain fused modal features of different modalities, wherein the multimodal fusion model regards the data of different modalities in the multimodal database as fields in an abstract space, and realizes information fusion of different modalities by simulating the dynamic evolution and interaction of different fields. The simulation of the dynamic evolution and interaction of different fields is expressed by the following formula:

[0005] ,in, is a field function, For time, is the partial differential symbol, is the diffusion coefficient, is the Laplace operator, Used to describe the smoothness of field functions in space; is the reaction coefficient, is the reaction function, The original modal features are processed by the Transformer encoder to obtain the fused feature tensor and obtain the general high-level features. The multi-task output head is designed according to the business needs of the target cold chain warehouse to process the high-level features.

[0006] Furthermore, the various types of sensors include: temperature and humidity sensors, gas concentration sensors, air quality sensors, light sensors, infrared cameras, 3D ToF cameras or lidars, and UWB positioning base stations.

[0007] Furthermore, temperature and humidity sensors are used to monitor and record ambient temperature and humidity data inside the warehouse; gas concentration sensors may include: carbon dioxide sensors and ethylene sensors, which are used for environmental safety monitoring and cargo status assessment in high-value fruit and vegetable storage areas; air quality sensors are used to monitor the air cleanliness in cold chain warehouses; light sensors are used to monitor the light intensity inside the warehouse, assisting in energy-saving management and optimizing operational lighting; infrared cameras are used to measure the surface temperature of cargo over a long distance and in a non-contact manner, while quickly identifying abnormal temperature areas within the warehouse and visually displaying the temperature distribution of the entire warehouse.

[0008] Furthermore, 3D ToF cameras or lidars are used to perceive the status and behavior within the warehouse, measure the volume of goods entering the warehouse, detect stacking status, update the storage location in real time, detect channel blockage, and identify the behavior of personnel / unmanned forklifts; UWB positioning base stations are used to support high-precision indoor positioning, realizing real-time location tracking of unmanned forklifts, staff, high-value turnover boxes / pallets, path optimization, and operational efficiency analysis.

[0009] Furthermore, the multimodal database includes: sensor data, inventory records and external data of the target cold chain warehouse; the inventory records are product and storage status information in the target cold chain warehouse, which are used to achieve refined management of the inventory; the external data are the macro environment and market background information data of the target cold chain warehouse, which are used for the AI ​​model to understand the impact of external factors on the operation of the cold chain warehouse.

[0010] Furthermore, inventory records include: basic product information, product name and description, batch or serial number, production date, packaging date, shelf life, recommended storage temperature-humidity range of goods, packaging size and weight, inventory status information, current inventory quantity, storage location information, out / in stock time, supplier information, and customer order association.

[0011] Furthermore, external data may include: weather data, market and demand data, and supply chain and logistics data.

[0012] Furthermore, a multimodal fusion model is constructed through the following steps: first, the heterogeneous multimodal data in the multimodal database are mapped into a unified mathematical space to provide a unified spatial dimension embedding for subsequent field modeling; then, based on partial differential equations and combined with the unified spatial dimension embedding, a field dynamics equation is constructed to describe how information of different modalities propagates, interacts and evolves in this unified space, and the coupling equation is derived according to the actual application scenario of the target cold chain warehouse.

[0013] Furthermore, mapping the heterogeneous multimodal data in the multimodal database into a unified mathematical space includes: performing feature extraction and dimensionality reduction through deep learning models to capture the intrinsic semantics of each modality; projecting the feature vectors of all modalities into the same d-dimensional space through a fully connected layer, and mapping the data of each modality into this d-dimensional Euclidean space to obtain the orthogonal coordinates of different modalities in the space; and using orthogonal coordinates to distinguish the characteristics of different data themselves and the modal space position of the data, providing a spatial dimension for subsequent model construction.

[0014] Furthermore, the reaction function is expressed as follows:

[0015] ,in, is the specific number of modes, is the total number of modes; is the weight function, is the interaction coefficient, which is used to control the strength of the cross-attention mechanism; It is a cross attention mechanism.

[0016] Furthermore, the weight function is constructed using the Gaussian kernel function and is expressed as follows:

[0017] ,in, is the symbol of the natural exponential function; is the standard deviation of the Gaussian kernel, which determines the expansion range of the weight function; is the modal point.

[0018] Furthermore, the cross attention mechanism is expressed as follows:

[0019] ,in, is the number of modal points in the neighborhood, is the attention weight, which indicates the degree of association between the modality position and its domain, is the number of modes, For location field.

[0020] Furthermore, the attention weight is calculated by the softmax function and is expressed as follows:

[0021] ,in, For location The dimensional vector obtained by linear transformation represents the position The characteristics of the modal points; For location Neighborhood The dimension vector obtained by linear transformation represents the neighborhood The characteristics of the modal points; is the transpose symbol of the matrix, which is used to ensure that dot product operations can be performed between vectors.

[0022] Furthermore, the Transformer encoder is an improved Transformer encoder, which generates queries and keys through the fused feature tensor, ensures that the attention mechanism operates based on the fused information, and calculates the similarity between all queries and all keys through the queries and keys, thereby determining which parts should pay attention to each other, and the similarity results are scaled by the dimension scaling factor to stabilize training; subsequently, different masks are introduced according to different tasks, forcing the Transformer encoder to perform attention calculations only within the range related to the mask.

[0023] Furthermore, the improved Transformer encoder can be expressed as follows:

[0024] ,in, is the query in the attention mechanism, is the key in the attention mechanism, is the value in the attention mechanism; is the Softmax function, used to normalize the weights; is the query weight matrix, is the key weight matrix, is the dimension scaling factor used to stabilize numerical calculations, is the mask matrix.

[0025] On the other hand, the present invention provides a cold chain warehouse system based on a multimodal AI large model, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, it implements any of the above-mentioned methods for constructing a cold chain warehouse system based on a multimodal AI large model.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] 1. This invention achieves effective integration of multi-source heterogeneous data by arranging various types of sensors in cold chain warehouses and combining them with a multimodal database deployed in the cloud. This overcomes the limitation of traditional systems that rely only on a single data source, realizes comprehensive data collection and storage, and provides more comprehensive and multi-dimensional data support for subsequent intelligent analysis.

[0028] 2. The present invention uses a multimodal fusion model to process data in a multimodal database, cleverly applying field theory and diffusion-reaction equations in traditional physics to the multimodal data fusion of cold chain warehouses, achieving effective fusion and correlation analysis of different modal information, and breaking through the limitations of traditional data processing methods in processing multimodal data.

[0029] 3. The present invention uses an improved Transformer encoder to process fusion features and combines it with a multi-task output head to achieve accurate prediction of future demand, overcoming the inefficiency and error-proneness of manual periodic inspection records in the existing technology, significantly improving the efficiency and accuracy of inventory management, and at the same time enabling the transformation of the cold chain warehouse system from passive response to active prediction and intelligent management. Through the application of multimodal AI large models, intelligent and automated operation of cold chain warehouses is realized, effectively reducing labor costs, and improving operational efficiency and product safety levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0031] Figure 1 This is a flow chart of the method provided in Example 1 of the present invention.

[0032] Figure 2 This is a timing diagram of the method provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0034] Example 1

[0035] This embodiment discloses a method for constructing a cold chain warehouse system based on a multimodal AI large model

[0036] Figure 1 The flowchart of the cold chain warehouse system construction method in this embodiment is shown. It can be seen from the figure that this embodiment includes the following steps:

[0037] Step 1: Deploy multiple types of sensors in the target cold chain warehouse and integrate sensors of the same type into a common sensor node. The sensor node inputs the collected data into the edge gateway for preprocessing and sends the preprocessed data to the cloud.

[0038] Specifically, in the unique environment of a cold chain warehouse, sensor selection and deployment must fully consider factors such as low temperature, high humidity, and stacked goods to ensure data accuracy and system stability. In this embodiment, sensors may include: temperature and humidity sensors, gas concentration sensors, air quality sensors, light sensors, infrared cameras, 3D Time of Flight cameras / LiDAR, and UWB positioning base stations.

[0039] Temperature and humidity sensors are used to monitor and record ambient temperature and humidity data within the warehouse. Gas concentration sensors, such as carbon dioxide and ethylene sensors, can be used for environmental safety monitoring and cargo status assessment in high-value fruit and vegetable storage areas. Air quality sensors are used to monitor air cleanliness in cold chain warehouses, particularly in cold chain processes such as food and pharmaceuticals, which have strict hygiene requirements. Light sensors are used to monitor light intensity within the warehouse, assisting with energy-saving management and optimizing operational lighting. Infrared cameras are used to measure the surface temperature of goods remotely and contactlessly, quickly identifying areas of abnormal temperature within the warehouse and visually displaying the temperature distribution throughout the warehouse.

[0040] 3D Time of Flight cameras and LiDAR are used for warehouse status and behavior perception, measuring incoming cargo volume, detecting stacking status, real-time location updates, detecting aisle congestion, and identifying personnel and unmanned forklift behavior. UWB positioning base stations support high-precision indoor positioning, enabling real-time location tracking of unmanned forklifts, personnel, and high-value containers and pallets, as well as route optimization and operational efficiency analysis.

[0041] Step 2: Build a cloud computing platform in the cloud. The cloud receives the data sent by the edge gateway and stores it in the multimodal database of the target cold chain warehouse corresponding to the cloud computing platform.

[0042] The target cold chain warehouse's multimodal database contains not only sensor data from the target cold chain warehouse, but also inventory records and external data. This multimodal database provides data support, providing more comprehensive, multi-dimensional insights for subsequent intelligent analysis, leading to more accurate decision-making.

[0043] Specifically, in this embodiment, inventory records contain rich product and storage status information to support refined management and forecasting. Inventory records can include basic product information, product name and description, batch / serial number, production date / packaging date, shelf life / expiration date, recommended storage temperature and humidity ranges, package size and weight, inventory status information, current inventory quantity, storage location information, inbound / outbound times, supplier information, and customer order associations.

[0044] Basic product information includes: SKU / product code, which uniquely identifies each product; product name and description, which detail the product type; batch / serial number, which is crucial for traceable items (such as pharmaceuticals and high-value foods); production / packaging date, which identifies the product's initial condition; and shelf life / expiration date, which is one of the most critical pieces of information for products stored in cold chain warehouses and directly impacts storage strategies and outbound priorities.

[0045] In this embodiment, external data can provide a broader macro-environmental and market context for cold chain warehouses, helping the AI ​​model understand the impact of external factors on demand, energy consumption, and operations. Specifically, external data can include weather data, market and demand data, historical sales data, holiday / promotional information, seasonal trends, supply chain and logistics data, etc.

[0046] The inventory records and external data in the multimodal database, combined with the sensor data in the target cold chain warehouse, will be input into the multimodal fusion model deployed in the cloud, so as to achieve accurate demand forecasting. By combining historical shipments, holidays, weather, market trends, etc., the demand for different products in the future can be predicted. Warehouse location optimization, based on the product shelf life, storage requirements, shipment frequency, and current warehouse temperature / humidity data, intelligently recommends the best storage location. And combined with external weather, inventory, refrigeration equipment operating status, etc., intelligently adjust the refrigeration strategy to reduce energy consumption, etc. The method disclosed in this embodiment integrates these rich data to enable the cold chain warehouse system to transform from passive response to active prediction and intelligent management, thereby greatly improving operational efficiency and product safety.

[0047] Step 3: Process the data in the multimodal database through the multimodal fusion model in the cloud computing platform to obtain a fusion modal feature of different modalities.

[0048] The multimodal fusion model in this embodiment cleverly applies the field theory and diffusion-reaction equations in traditional physics to the multimodal data fusion of cold chain warehouses. Its core idea is to regard the collected data of different modalities as fields existing in an abstract space. By simulating the dynamic evolution and interaction of these fields, it realizes the information fusion of different modalities in the cold chain warehouse and obtains a fusion modal feature that integrates different modalities.

[0049] Specifically, in this embodiment, the multimodal fusion model can be constructed through the following steps:

[0050] 1) Mapping heterogeneous multimodal data in a multimodal database into a unified mathematical space to provide a unified spatial dimension embedding for subsequent field modeling.

[0051] Building a multimodal fusion model requires converting raw heterogeneous modal data (e.g., images, time series, tables, text, etc.) into a unified, actionable numerical representation and assigning them a location in an abstract space. This is accomplished by converting data from different sources into a unified d-dimensional feature vector and associating it with predefined modal coordinates.

[0052] Specifically, deep learning models are first used to extract features and reduce dimensionality to capture the intrinsic semantics of each modality. For example, in this embodiment, a pre-trained CNN model can be used to extract and reduce features for the image modality; an LSTM model can be used to extract and reduce features for the temperature modality; an MLP model can be used to extract and reduce features for the inventory modality; and an NLP model can be used to extract and reduce features for text.

[0053] After the extraction is completed, the feature vectors of all modalities are projected into the same d-dimensional space R through the fully connected layer d And map the data of each mode into this d-dimensional Euclidean space to obtain the orthogonal coordinates of different modes in this space.

[0054] For example, in this embodiment, four modal data, namely, image, temperature, inventory record, and external data, are used, so the constructed Euclidean space is a 4-dimensional Euclidean space, which can be expressed as: ,in, is Euclidean space; is the modal space. In this embodiment, the orthogonal coordinate positions of different modalities can be expressed as: ,temperature , inventory records , external data , that is to say, the coordinate position of each modality data in four-dimensional space ensures that different modalities can be compared and interacted in a unified space.

[0055] In other words, these coordinates can be understood as modal identities or location markers. By assuming that different modes are independent in some abstract modal space, yet interconnected through field diffusion and reaction mechanisms, orthogonal coordinates can be used to distinguish the characteristics of the data from its modal space location, providing a spatial dimension embedding for subsequent model construction.

[0056] 2) Based on a unified spatial dimension embedding, a field dynamics partial differential equation is constructed to describe how information from different modalities propagates, interacts, and evolves in this unified space. Based on the specific application scenarios of cold chain warehouses, specific coupling equations that meet the actual operational objectives of cold chain warehouses are derived from the field dynamics partial differential equations and numerically solved. For example, the finite difference method can be used to transform the continuous partial differential equation into a discrete algebraic system of equations, dividing the multidimensional modal space into discrete grid points to achieve spatial discretization and realize numerical solution.

[0057] Specifically, in this step, by introducing a continuous field function , which is used to describe any position in the modal space and any time Based on the fusion features on the field function, a unified mathematical framework is constructed to describe the interaction and information diffusion between different modes. The mathematical framework can be expressed as follows:

[0058] ,

[0059] in, is a field function, For time, is the partial differential symbol, is the diffusion coefficient, which is used to control the speed and range of information propagation in the modal space; is the Laplace operator, Used to describe the smoothness of a field function in space. The Laplace value of a point measures the difference between the point and the average value of the surrounding points. A positive value indicates that the point is lower than the surrounding average, and a negative value indicates that it is higher. During the diffusion process, information always flows from high-concentration areas to low-concentration areas, achieving smoothing and homogenization, promoting information sharing and fusion between adjacent modes, and achieving spatial consistency. is the reaction coefficient, which is used to control the intensity of the influence of the reaction function on the field evolution; is the reaction function, is the original modal feature.

[0060] It should be noted that the unified mathematical framework is an abstract general framework that assumes that there is a single fusion field in a unified, abstract multimodal space. , this field diffuses and interacts in all modal dimensions. Its purpose is to fuse the characteristics of all modalities into a unified field, and in this abstract modal space, information evolves dynamically through diffusion and smoothing as well as complex reactions. Then, based on this single fusion field, different modalities can be selected from the unified abstract field according to the specific situation of the target cold chain warehouse. In other words, the fusion field All modalities are considered a single continuum in a higher-dimensional space, and the behavior of this continuum is described by a universal PDE, presenting a macro, unified perspective on multimodal fusion. Different modalities are then selected based on the actual task requirements, and a partial differential equation is constructed for each modality to define an independent field. These independent equations are linked through explicit coupling terms. Each equation and term in the explicit coupling term corresponds to the physical processes of a specific modality and the specific interactions between them. This approach presents a more micro, specific, and domain-knowledge-driven perspective on multimodal fusion.

[0061] Specifically, in this embodiment, the reaction function can be expressed by the following formula:

[0062] ,

[0063] in, is the specific number of modes, is the total number of modes; is the weight function, is the interaction coefficient, which is used to control the strength of the cross-attention mechanism; It is a cross attention mechanism.

[0064] In this embodiment, the weight function can be constructed using a Gaussian kernel function, as shown in the following formula:

[0065] ,

[0066] in, is the symbol of the natural exponential function; is the standard deviation of the Gaussian kernel, which determines the expansion range of the weight function; is the modal point.

[0067] In this embodiment, the cross attention mechanism is as follows:

[0068] ,

[0069] in, is the number of modal points in the neighborhood, is the attention weight, which indicates the degree of association between the modal position and its domain. The attention weight can be calculated by the softmax function and is expressed as follows:

[0070] ,

[0071] in, For location The dimensional vector obtained by linear transformation represents the position The characteristics of the modal points; For location Neighborhood The dimension vector obtained by linear transformation represents the neighborhood The characteristics of the modal points; is the transpose symbol of the matrix, which is used to ensure that dot product operations can be performed between vectors.

[0072] It should be noted that, in the reaction function shown in this embodiment, Is the fidelity term, used to ensure that at each modal point Nearby, current field value Can be pulled toward the original modal features ,This is like an anchoring mechanism, preventing information of different modalities from completely deviating from the original data during ,the process of diffusion and interaction; is the interaction term, used to pass the cross attention mechanism To describe the correlation between different modalities, thereby promoting high-order and selective information exchange between modalities.

[0073] Through the diffusion and reaction of the unified mathematical framework, the original modal features of the same modality are fully integrated with its original modal information to form a new fused modal feature that can achieve context-awareness. The fused modal feature is no longer an isolated modal feature, but an enhanced feature that includes the coupling effects between different modalities.

[0074] 3) This step may also include deriving specific coupling equations from a unified mathematical framework based on specific cold chain warehouse application scenarios. In this embodiment, four modalities are used: image, temperature, inventory records, and external data. Therefore, the following set of coupling equations can be constructed:

[0075] ,

[0076] in, is the image feature field, which is the feature representation of pixels in an image or video; is the temperature field; is the inventory status field; It is an external data field, representing external information related to the cold chain warehouse; is the image diffusion coefficient, which indicates the smoothness of the image information in space; is the stock diffusion coefficient, which is used to control the propagation speed of the temperature effect of different stocks; is the temperature, is the attenuation coefficient, which indicates the attenuation or disappearance of the image feature field; The spatially dependent heat transfer coefficient indicates that the heat conduction rate is different at different locations. For example, the heat conduction rate may be different on the shelf and in the aisle, which enables the model to capture the temperature dynamics of different physical areas. The source term of the image modality represents the raw image data continuously injected from the outside (such as real-time images or video frames from an infrared camera), which continuously provides new and unprocessed visual information to the system and drives the evolution of the entire system; is the source term of the temperature mode, representing the real-time temperature data input from devices such as temperature and humidity sensors; The source item of the inventory mode comes from the real-time inventory records of the system; is the activation function, which can be sigmoid or ReLU, used to introduce nonlinearity; is the weight matrix embedded in the external factors; is an external factor, Embedding representations for external factors (such as weather, supply chain, etc.) usually involves converting raw text or numerical data into vectors through some kind of embedding model (such as word embedding); is the bias term for external factor embedding.

[0077] is a temperature coupling function, which can be a Gaussian function. Only when the temperature is close to the preset optimal temperature of the environment, the coupling strength of the visual information and the temperature information is the largest; if the temperature deviates too far from the preset optimal temperature of the environment, the value of the coupling function will approach 0, indicating that the correlation between temperature and visual information is weakened. The Gaussian function can be used to adapt to different products in the cold chain by adjusting the standard deviation of the Gaussian kernel. The Gaussian kernel determines the sharpness of the function curve. The smaller the Gaussian kernel, the sharper the curve, indicating that the product is very sensitive to temperature changes and only has strong coupling within a very small temperature range; and the larger the Gaussian kernel, the flatter the curve, indicating a higher tolerance to temperature. In other words, by adjusting the Gaussian kernel, different coupling degrees can be easily adjusted. is the inventory coupling function, which can be in the form of a hyperbolic tangent function, and is used to characterize that the impact of inventory on temperature is nonlinear. That is, when the inventory is lower than a certain critical threshold, it may cause the fluctuation of ambient temperature to intensify. The external influence function is a piecewise linear function that can be constructed using the ReLU activation function. It is used to characterize that external influences only take effect when they exceed a certain threshold. That is to say, when a certain preset threshold is exceeded, the activation function will be activated and affect the inventory, thereby making the inventory dynamics affected by external factors.

[0078] It should be noted that the core concept of the coupled equations in this embodiment is to view multimodal data as interacting fields, with each modality (visual, temperature, inventory, external data) represented by a field variable. The entire set of equations constitutes a dynamic coupled system, describing the evolution of different fields in time and space. This evolution is driven by three primary processes: diffusion, which describes how information or features propagate and disperse in space; reaction, which describes how different modalities interact and influence each other, and is the core of multimodal fusion; and source term, which describes how new information or data is injected into the system from the outside. Each equation in the system follows the basic pattern of diffusion + reaction + source term. In this way, this coupled equation system cleverly integrates information from different modalities and at different spatial and temporal scales under the constraints of physical laws, thereby outputting a unified representation that is more physically meaningful and contextually relevant.

[0079] Step 5: Through the above steps, data from different modalities and different spatiotemporal scales are simulated to diffuse and react in the abstract field, and finally a unified, context-aware feature representation is generated, i.e., a fused feature tensor. The fused feature tensor is processed using the Transformer encoder improved by the temperature mask, and the predicted future demand is output through the multi-task output head. Specifically, this step may include the following sub-steps:

[0080] 1) First, the Transformer encoder requires a sequence of uniform dimensions as input, and this sequence should contain content information and position information. The initial input sequence of the Transformer encoder can be calculated by the following formula: :

[0081] ,

[0082] in, Layer normalization is used to stabilize training; is the weight matrix of fusion features; is the weight matrix of position encoding; Encode the position.

[0083] The purpose of the above formula is to provide a rich context information for the Transformer encoder (through ) and has clear location information (via ) is the initial input. and A linear projection is performed to ensure that both can adapt to the internal dimensions of the Transformer. Then, the two are added together so that the model can utilize both key information at the same time. Finally, layer normalization is performed to stabilize and optimize the training of the model.

[0084] 2) The query, key, and value of the attention mechanism are derived from the initial input sequence, and the fused features are further nonlinearly transformed and interacted with through the improved Transformer encoder. In this embodiment, the improved Transformer encoder is an encoder that adds a temperature mask matrix, which can be expressed as follows:

[0085] ,

[0086] in, is the query in the attention mechanism, is the key in the attention mechanism, is the value in the attention mechanism; is the Softmax function, used to normalize the weights; is the query weight matrix, is the key weight matrix, is a dimensionality scaling factor, usually the dimension of the key, used to stabilize numerical calculations, is the temperature mask matrix.

[0087] The temperature mask matrix can be expressed as follows:

[0088] ,

[0089] in, is the temperature at position i, is the temperature at position j, is the preset temperature threshold. In other words, this formula means: Condition, generate the temperature mask matrix matrix, and fill the attention score positions that do not meet the condition as .

[0090] It should be noted that the improved Transformer encoder in this embodiment uses the fused feature tensor to generate a query (via ) and keys (via ), ensuring that the attention mechanism operates based on fused information. In the above formula, the similarity between all queries and all keys is calculated using the query and key to determine which parts should be attended to. The resulting similarity is then scaled by a dimensionality scaling factor to stabilize training. Subsequently, a temperature mask is introduced to force the Transformer encoder to only focus on the temperature-relevant range, reflecting domain knowledge in the specific application scenario of cold chain warehouses. The softmax function converts the similarity scores into a probability distribution, namely attention weights, which indicate the contribution of each value to the final output. Finally, these attention weights are multiplied by the value matrix V and weighted summed to produce the final attention output, which intelligently aggregates the relevant information from the input features. In other words, after the improved Transformer encoder processes the initial input sequence, it outputs the final general high-level features.

[0091] 3) Using multi-task output heads, generic high-level features are designed for different tasks (demand, position, and decay). Different types of targets are specifically extracted and predicted from the generic high-level features output by the Transformer. Specifically, in this embodiment, task-specific output heads can include demand prediction, position prediction, and decay prediction.

[0092] Demand forecasting is the predicted future demand for a specific entity (e.g., a warehouse location, a product). Demand forecasting can be expressed as follows:

[0093] ,

[0094] in, For demand forecasting; A multi-layer perceptron is used for the final classification or regression task; is the [CLS] labeled output of the last layer of the Transformer encoder.

[0095] It should be noted that the above formula represents the use of a multilayer perceptron as a classifier or regressor for downstream tasks, receiving the aggregated features extracted by the Transformer, and learning how to map them to the final demand forecast value.

[0096] Location prediction is used to predict the location of a specific event, the location where an entity should be placed, or the optimal storage location for a resource. Location prediction can be expressed as follows:

[0097] ,

[0098] in, For position prediction, It is a graph neural network used to process data with graph structure; is the output of the last layer of the Transformer encoder; is the Hadamard product symbol, which means element-by-element multiplication; is the priority mask matrix used to emphasize certain locations.

[0099] It should be noted that the above formula represents the use of GNN as the output head, representing that the location information has a certain graph structure. For example, there may be a connection relationship between storage locations (adjacent, path reachable, etc.), or there may be a network relationship between entities in space. GNN can effectively capture the relationship between nodes (for example, storage locations) and their neighbors, thereby better inferring precise location information. Combined with priority masks, GNN can selectively aggregate information from high-priority areas.

[0100] Attenuation prediction is used to predict the attenuation of a physical quantity (e.g., inventory, product quality) over time. Attenuation prediction can be expressed as follows:

[0101] ,

[0102] in, For attenuation prediction, It is a bidirectional long short-term memory network used to capture time series information; is a natural constant; is the attenuation coefficient, is the time decay factor; is the output of the i-th position of the last layer of the Transformer encoder.

[0103] It should be noted that the above formula introduces the physical model of time decay into the prediction, which means that the prediction result depends not only on the features learned by the bidirectional long short-term memory network, but also on the time distance. The farther the data point is from the current time, the exponentially attenuated its influence will be.

[0104] In the method disclosed in this embodiment, the Transformer encoder performs well in processing sequences and capturing long-distance dependencies. The multimodal fusion model in the previous step completes the fusion of multimodal data at the physical level (based on the dynamic evolution of diffusion and reaction), while the Transformer encoder in this step is responsible for semantic refinement and multi-task adaptation. The two are combined to complement each other, and the fusion features are processed by the Transformer encoder to generate a more robust and expressive unified representation. The unified representation finally generated can be used for a variety of tasks, for example, to realize demand forecasting of cold chain warehouses (which may depend on inventory, external weather, and even visual features of goods), storage location optimization (considering temperature, inventory, visual recognition, etc.), and energy consumption recommendations (temperature, inventory, etc.). In order to better illustrate the overall construction sequence in this embodiment, Figure 2 FIG. 2 shows a timing diagram of the method of this embodiment.

[0105] Example 2

[0106] This embodiment discloses a method for constructing a cold chain warehouse system based on a multimodal AI large model. The main steps in this embodiment are the same as those in Example 1. The differences from Example 1 are: the specific coupling equations derived from the unified mathematical framework based on the specific cold chain warehouse application scenario; the improvements to the Transformer encoder and the design of the multi-task output head.

[0107] In this embodiment, based on a unified mathematical framework, to achieve more refined optimization of the cold chain warehouse environment, we introduce energy consumption modalities for precisely controlling cold chain warehouse energy consumption; equipment health modalities for characterizing the impact of cold chain warehouse equipment outages and maintenance costs; operational process modalities for reflecting the efficiency of cold chain warehouse operations such as warehousing, outbound delivery, and picking; and order flow modalities for characterizing cold chain warehouse workload and resource allocation. By abstracting these modalities from the unified mathematical framework, we achieve more refined cold chain warehouse management optimization.

[0108] In this embodiment, by assuming that these fields evolve primarily in time, and that some modes may be related to physical space or logical operation space, and assuming that the characteristics of all modes have been embedded in the d-dimensional feature space, the following coupled equations can be constructed:

[0109] ,

[0110] in, The energy consumption mode represents the real-time energy consumption intensity or density of different areas in the warehouse at a certain time. High values ​​indicate high energy consumption, and low values ​​indicate low energy consumption. The equipment health field represents the health status of various equipment in the warehouse (refrigeration equipment, forklifts, conveyor belts, etc.) in space and time. A high value indicates health, and a low value indicates deterioration or failure. is the operation process field, which represents the efficiency or smoothness of different operation links (such as warehousing, picking, and outbound, which can be abstracted as points in the operation process space) at time t. A high value indicates high efficiency, and a low value indicates low efficiency. It represents the real-time order processing volume or the number of pending orders at time t. It is usually regarded as a global variable and is not strongly dependent on the physical space. Its data comes from the order management system.

[0111] is the energy diffusion coefficient, Used to simulate the diffusion or mutual influence of energy consumption in a physical space. For example, the energy consumption in the area around a high-energy-consuming device will also be relatively high, or air conditioning leakage will cause energy consumption to spread. is the coupling strength coefficient, which indicates the intensity of the impact of equipment health on energy consumption. It is used to represent the inhibitory effect of equipment health on energy consumption. Healthy equipment is usually more energy-efficient, thereby reducing energy consumption. Conversely, equipment failure or aging will increase energy consumption, so this is a negative reaction term. is the coupling intensity coefficient of order flow, which indicates the intensity of the impact of order flow on energy consumption; It indicates the promoting effect of order flow on energy consumption. The larger the order volume and the more frequent the warehouse activities (lighting, forklifts, cold storage door opening, etc.), the higher the energy consumption. It is the source item of the energy consumption mode, and its data comes directly from the data injection of real-time energy sensors.

[0112] is the equipment health conduction coefficient, The device health conduction item is used to simulate the spatial impact of the device health status. For example, a refrigeration unit failure may affect the cooling effect of adjacent areas, thereby affecting the health of other equipment in the area. is the coupling intensity coefficient of energy consumption, It is the energy consumption coupling term, which indicates the inhibitory effect of energy consumption on equipment health. Long-term high energy consumption (which may mean overload operation) will lead to the decline of equipment health. is the coupling strength coefficient of the operation process, It is the operation process coupling item, which indicates the inhibitory effect of high-load operation processes on equipment health. Frequent operation operations (such as frequent opening of cold storage doors and high-intensity use of forklifts) will accelerate equipment wear. It is the source item of the device health mode, and its data source comes from the real-time health monitoring data injection of the device.

[0113] is the dynamic coefficient of the operation process, It is a dynamic response item of the operation process, which is used to simulate the inertial or elastic response of the operation process to changes. It means that the change of process efficiency depends not only on the current state but also on the speed of change. For example, a sudden drop in process efficiency may lead to more drastic efficiency fluctuations in the future. is the coupling strength coefficient of order flow, It is the order flow coupling item, which is used to express the promoting effect of order flow on the efficiency of the operation process. Reasonable order flow can improve the synergy of the process, but overload will lead to reduced efficiency. is the coupling strength coefficient of the equipment health, This is the equipment health coupling item, which represents the inhibitory effect of equipment health on the efficiency of the operation process. Equipment failure or performance degradation will directly affect the smoothness of the operation process. is the source term of the operation mode, For different operation links, the data source of this source item is the real-time operation data injection from warehouse management.

[0114] is the order flow dynamic coefficient, It is the dynamic term of order flow itself, indicating that the rate of change of order flow is also affected by its own current rate of change, thereby simulating the inertia effect of order peaks or troughs. is the coupling strength coefficient of the operation process, It is the operation process coupling item, which represents the feedback effect of operation process efficiency on order flow. An efficient operation process can process orders faster, thereby accelerating the turnover of order flow or reducing backlog. This is the source item of the order mode, and the data source of this source item is the real-time order data injection from the order management system.

[0115] It should be noted that the equation construction logic of this embodiment still adheres to the core concept of the unified mathematical framework in Example 1. By selecting different modes to abstract as fields, energy consumption, equipment health, operating processes, and order flow are all considered fields that can dynamically change in time or space. The time derivative of each modal field is used to represent its change over time. Diffusion / dynamic terms related to the modal's own characteristics are introduced to simulate the propagation of these modal information in physical space. Each equation in the equation system disclosed in this embodiment includes nonlinear coupling terms with other modal fields, which are key to the model's ability to capture intermodal interactions. Different coupling coefficients are used to determine the strength and direction (promotion or inhibition) of the mutual influence between different modes. Real-time data injection (source term) is fully considered, and the dynamic evolution of the model is continuously driven by the real-time information injection from various raw data sources.

[0116] In this way, the core equations in this embodiment enable real-time monitoring of cold chain warehouses, dynamically understanding the real-time status of energy consumption, equipment, operations, and orders. Correlation analysis can also be performed to reveal the underlying interactions between various modalities (for example, how equipment aging affects energy consumption, or how order peaks affect operational efficiency). This enables predictive insights, enabling the prediction of future energy consumption trends, equipment failure risks, process bottlenecks, or order backlogs based on current and historical dynamics. This provides cold chain warehouse managers with a decision-making basis based on physics and business logic, allowing them to identify operational bottlenecks, optimize personnel and equipment scheduling, and rationally allocate human and equipment resources based on dynamic predictions of order flow and operational efficiency.

[0117] Through the coupled equations in this embodiment, the fusion characteristic tensors of different modes are obtained: In this embodiment, the fused feature tensor The fusion characteristics of the four modes obtained by calculating the coupled equations in this embodiment are: 、 、 、 , spliced ​​together to obtain, among which, is the fusion feature of energy consumption mode, For the fusion characteristics of the operation process field, For the fusion characteristics of the equipment health field, is the fused feature of the pending order quantity. The fused feature tensor is then used to calculate the initial input sequence for the Transformer encoder. The query, key, and value of the attention mechanism are derived from the initial input sequence. The fused features are further nonlinearly transformed and interacted with using the improved Transformer encoder. In this embodiment, the improved Transformer encoder incorporates a job-stage mask matrix and can be expressed as follows:

[0118] ,

[0119] in, is the job stage mask matrix, which can be expressed as follows:

[0120] .

[0121] Since the purpose of the coupled equations in this embodiment is to improve the operating efficiency of the cold chain warehouse, the design of a new multi-tasking output head can include equipment failure prediction, operation process bottleneck identification, and energy consumption optimization suggestions.

[0122] Among them, equipment failure prediction can use aggregated features to predict the probability of equipment failure within a certain time window in the future.

[0123] To identify bottlenecks in the operation process, GNN can be used to capture the relationship between different operation links and combine the information of the operation process field to identify potential efficiency bottleneck areas.

[0124] Energy consumption optimization suggestions can provide energy consumption reduction suggestions (such as when to adjust the cooling temperature and which equipment needs maintenance) based on the fused energy consumption field and other modal information.

[0125] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a cold chain warehouse system based on a multimodal AI large model, characterized in that: The cold chain warehouse system construction method includes: Deploy various types of sensors in the target cold chain warehouse. The edge gateway pre-processes the collected data and sends the processed data to the cloud. The cloud receives the data from the edge gateway and stores the data in the multimodal database of the target cold chain warehouse on the cloud computing platform deployed in the cloud; The multimodal fusion model of the cloud computing platform is used to process the data in the multimodal database to obtain the fusion modal features of different modalities. The multimodal fusion model regards the data of different modalities in the multimodal database as fields in an abstract space, and realizes the information fusion of different modalities by simulating the dynamic evolution and interaction of different fields. The dynamic evolution and interaction of the simulated different fields are expressed by the following formula: ， in, is a field function, For time, is the partial differential symbol, is the diffusion coefficient, is the Laplace operator, Used to describe the smoothness of field functions in space; is the reaction coefficient, is the reaction function, is the original modal feature; The Transformer encoder is used to process the fused feature tensor to obtain common high-level features. A multi-task output head is designed to process the high-level features according to the business needs of the target cold chain warehouse.

2. The method for constructing a cold chain warehouse system based on a multimodal AI large model according to claim 1 is characterized in that: The various types of sensors include: temperature and humidity sensors, gas concentration sensors, air quality sensors, light sensors, infrared cameras, 3D ToF cameras or lidars, and UWB positioning base stations.

3. The method for constructing a cold chain warehouse system based on a multimodal AI large model according to claim 1 is characterized in that: The multimodal database includes: sensor data, inventory records and external data of the target cold chain warehouse; The inventory record is the product and storage status information in the target cold chain warehouse, which is used to achieve refined management of the inventory; The external data refers to the macro-environment and market background information data of the target cold chain warehouse, which is used by the AI ​​model to understand the impact of external factors on the operation of the cold chain warehouse.

4. The method for constructing a cold chain warehouse system based on a multimodal AI large model according to claim 1 is characterized in that: The multimodal fusion model is constructed through the following steps: First, the heterogeneous multimodal data in the multimodal database are mapped into a unified mathematical space to provide a unified spatial dimension embedding for subsequent field modeling; Then, based on partial differential equations and combined with unified spatial dimension embedding, a field dynamics equation is constructed to describe how information of different modes propagates, interacts and evolves in this unified space, and the coupling equation is derived according to the actual application scenario of the target cold chain warehouse.

5. The method for constructing a cold chain warehouse system based on a multimodal AI large model according to claim 1 is characterized in that: The reaction function is represented by the following formula: ， in, is the specific number of modes, is the total number of modes; is the weight function, is the interaction coefficient, which is used to control the strength of the cross-attention mechanism; It is a cross attention mechanism.

6. The method for constructing a cold chain warehouse system based on a multimodal AI large model according to claim 5 is characterized in that: The weight function is constructed using a Gaussian kernel function and is expressed by the following formula: ， in, is the symbol of the natural exponential function; is the standard deviation of the Gaussian kernel, which determines the expansion range of the weight function; is the modal point.

7. The method for constructing a cold chain warehouse system based on a multimodal AI large model according to claim 5 is characterized in that: The cross-attention mechanism is expressed as follows: ， in, is the number of modal points in the neighborhood, is the attention weight, which indicates the degree of association between the modality position and its domain, is the total number of modes, For location field.

8. The method for constructing a cold chain warehouse system based on a multimodal AI large model according to claim 7 is characterized in that: The attention weight is calculated by the softmax function and is expressed as follows: ， in, For location The dimensional vector obtained by linear transformation represents the position The characteristics of the modal points; For location Neighborhood The dimension vector obtained by linear transformation represents the neighborhood The characteristics of the modal points; is the transpose symbol of the matrix, which is used to ensure that dot product operations can be performed between vectors.

9. The method for constructing a cold chain warehouse system based on a multimodal AI large model according to claim 1 is characterized in that: The Transformer encoder is an improved Transformer encoder, The improved Transformer encoder generates queries and keys through the fused feature tensors, ensuring that the attention mechanism operates based on the fused information. Calculate the similarity between all queries and all keys to decide which parts should pay attention to each other, and the similarity results are scaled by the dimension scaling factor to stabilize training; Different masks are introduced according to different tasks, forcing the Transformer encoder to perform attention calculations only within the range related to the mask.

10. A cold chain warehouse system based on a multimodal AI large model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the cold chain warehouse system construction method based on the multimodal AI large model as described in any one of claims 1 to 9.

Images