A store operation and equipment collaborative management system based on multimodal data fusion

Abstract

This invention discloses a store operation and equipment collaborative management system based on multimodal data fusion, belonging to the field of data processing technology. The system includes: a data acquisition module, a fusion module, a decision-making module, an equipment execution module, and an inventory execution module. This invention comprehensively acquires multimodal data of store operations by integrating data acquisition components such as logistics document interfaces and temperature and humidity sensors. It cleans heterogeneous data, removes outliers, and unifies formats. Then, it extracts the spatial features of structured data and the temporal features of unstructured data, and completes data fusion and association through an attention mechanism. Combined with the built-in decision rule base, it generates instructions to extend the freshness of refrigerated equipment. The data fusion process realizes deep association of various types of scattered data. The decision rule base provides accurate basis for instruction generation. The equipment execution module adjusts the equipment operating status and provides feedback, forming a complete control process, enabling the operation of refrigerated equipment to accurately match the needs of commodity storage.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a store operation and equipment collaborative management system based on multimodal data fusion. Background Technology

[0002] As the digital transformation of the retail industry deepens, the scale of store operations continues to expand, and the variety of products becomes increasingly rich, inventory status, equipment operation, and supply chain efficiency have become key factors affecting the core competitiveness of stores. Consumers' demands for product freshness and shopping experience are increasing. Stores need to achieve efficient resource allocation while ensuring product quality. The traditional decentralized management model is no longer able to cope with the complex needs of multi-stage collaboration. Integrated and intelligent operation management has become the industry development trend.

[0003] However, in existing technologies, store logistics data, inventory-related data, refrigeration equipment operation data, and shelf load-bearing data are mostly collected and managed by independent systems. The data is fragmented and heterogeneous in format, lacking a unified processing and integration mechanism, and cannot form effective correlation analysis. At the same time, existing technologies do not use targeted feature fusion methods to integrate multi-source data, the decision-making process lacks rule support based on historical operational data, and there is no feedback loop after execution. As a result, the operation of refrigeration equipment cannot be dynamically adjusted according to the status of goods, the inventory layout is difficult to match the shelf load-bearing status, the prediction of inventory replenishment nodes lacks data support, and the collaborative linkage between inventory, equipment, and supply chain cannot be achieved. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a store operation and equipment collaborative management system based on multimodal data fusion. This invention comprehensively acquires multimodal data of store operations by integrating data acquisition components such as logistics document interfaces, temperature and humidity sensors, and weight sensors. It cleans heterogeneous data, removes outliers, and unifies formats. Then, it extracts the spatial features of structured data and the temporal features of unstructured data, completes data fusion and association through an attention mechanism, and generates instructions to extend the freshness of refrigerated equipment by combining the built-in decision rule base. The data fusion process realizes deep association of various types of scattered data, the decision rule base provides accurate basis for instruction generation, and the equipment execution module adjusts the equipment operation status and provides feedback, forming a complete control process. This enables the operation of refrigerated equipment to accurately match the needs of commodity storage, ensures the quality of commodity storage, and achieves collaborative adaptation between data and equipment operation.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a store operation and equipment collaborative management system based on multimodal data fusion, comprising: Data Acquisition Module: Integrates logistics document interface, temperature and humidity sensor, weight sensor, shelf load sensor, energy consumption sensor and equipment status monitoring sensor to acquire multimodal data of store operation; Fusion module: It preprocesses the acquired multimodal data, performs feature extraction operations on structured and unstructured data respectively, and completes data fusion and association through attention mechanism to generate collaborative analysis data in a unified format; Decision module: Built-in decision rule library, based on collaborative analysis data and decision rule library, dynamically generates instructions to extend the freshness preservation time of refrigeration equipment and inventory layout optimization plan, and predicts inventory replenishment nodes to adjust the operation mode of refrigeration equipment; Equipment execution module: Based on the instruction to extend the preservation time of the refrigeration equipment, the operating parameters of the refrigeration equipment are adjusted in real time, and the operating status of the refrigeration equipment after adjustment is recorded and fed back to the fusion module; Inventory Execution Module: Based on the inventory layout optimization scheme, it sends adjustment reminder information to the work terminal, records the actual status of the inventory after adjustment, and feeds back to the fusion module.

[0006] Furthermore, the acquisition module connects to the logistics system via a logistics document interface to obtain unstructured text information from the logistics documents, including purchase order number, purchase cycle, arrival date, product name, product shelf life, and quantity received. Temperature and humidity sensors are deployed in the product storage area to collect temperature and humidity data of the storage environment. Weight sensors are installed under the shelf shelves to monitor the weight data of the corresponding product inventory on each shelf. Shelf load-bearing sensors are installed at the connection between the shelf uprights and beams to collect real-time load-bearing data of the entire shelf and each shelf. Energy consumption sensors are deployed in the power supply circuit of the refrigeration equipment to record real-time operating energy consumption. Equipment status monitoring sensors are integrated into the internal control unit of the refrigeration equipment to collect equipment operating status data such as the start / stop status of the refrigeration equipment and fault warning signals.

[0007] Furthermore, the fusion module performs data cleaning, outlier removal, and format unification preprocessing on the temperature and humidity data, commodity inventory and weight data, shelf load-bearing data, equipment energy consumption data, equipment operating status data, and logistics document text data in the multimodal data. This eliminates format differences and noise interference from heterogeneous data. Then, feature extraction is performed. The preprocessed structured sensor data is normalized to extract spatial features. The preprocessed unstructured text data is word-embedded and converted into vector form to extract temporal features. Finally, a fusion and association operation is performed. The extracted spatial and temporal features are associated and fused through an attention mechanism feature fusion formula to generate collaborative analysis data in a unified format.

[0008] Furthermore, in the fusion module, when fusing the extracted spatial and temporal features using the attention mechanism feature fusion formula, attention weights are first assigned. These attention weights are determined based on historical operational data statistics of the store, including past records of near-expiry product handling, equipment load change data, and supply chain response data. Then, the fusion of spatial and temporal features is completed using the attention mechanism feature fusion formula, which is as follows: ,in, For the synergistic features after fusion, For the first Spatial features, For the first One time-series feature, The total number of spatial features. The total number of time-series features. For the first Attention weights for each spatial feature relative to all temporal features For the first Attention weights of each temporal feature relative to all spatial features.

[0009] Furthermore, in the decision-making module, the decision rule base is built based on historical store operation data, including core rules such as product near-expiration judgment threshold, standard storage temperature and humidity for products, shelf load-bearing safety threshold, standard range of refrigeration equipment operating parameters, and inventory replenishment prediction threshold. Product shelf-life data and storage environment temperature and humidity data are extracted from the collaborative analysis data. The remaining shelf-life of the product is compared with the product near-expiration judgment threshold in the decision rule base. If it is lower than the near-expiration judgment threshold, the shelf-life extension time for the refrigeration equipment is first determined using the shelf-life extension formula. Then, based on the shelf-life extension time and the standard storage temperature and humidity of the product, the required temperature adjustment range for the refrigeration equipment is analyzed, and a shelf-life extension instruction containing temperature adjustment parameters is generated. The overall load-bearing data of the shelves and each shelf is extracted from the collaborative analysis data and compared with the shelf load-bearing safety threshold in the decision rule base. When it reaches 90% of the load-bearing safety threshold, the quantity of goods to be transferred is determined using the shelf goods transfer quantity formula, combined with real-time load-bearing data from other shelves, generating an inventory layout optimization plan.

[0010] Furthermore, in the decision-making module, the formula for extending the shelf life is: ,in, To extend the shelf life, the temperature adjustment range of the refrigeration equipment is determined based on the extended shelf life. The longer the extended shelf life, the closer the temperature adjustment range is to the lower limit of the standard storage temperature and humidity for the product. This refers to the remaining shelf life of the product. This refers to the total shelf life of the product. This represents the difference between the current storage environment's temperature and humidity and the product's standard storage temperature and humidity. The basic freshness retention factor for goods is set based on the characteristics of the product category. This is the environmental temperature and humidity correction factor, determined by the degree of deviation between the current storage environment temperature and humidity and the standard storage temperature and humidity of the product.

[0011] Furthermore, in the decision-making module, the formula for the amount of goods transferred from the shelf is: ,in, The quantity of goods to be transferred. The load-bearing safety threshold of the shelf, The current load-bearing capacity of the shelf. The load balancing coefficient is determined based on statistical analysis of historical shelf load optimization data from stores. The unit weight of the goods to be transferred is obtained by breaking down the inventory weight data of the goods.

[0012] Furthermore, the decision-making module extracts procurement cycle, delivery date, and current product inventory data from collaborative analysis data, compares the current product inventory with the inventory replenishment prediction threshold in the decision rule base, predicts the inventory replenishment node based on the procurement cycle, and adjusts the operating power and standby time of the refrigeration equipment before the replenishment node.

[0013] Furthermore, in the inventory execution module, the adjustment reminder information includes the name of the product to be adjusted, the current storage location, and the target storage location; the recorded actual status after inventory adjustment includes the completion status of product adjustment and the actual load data of each shelf after adjustment.

[0014] Compared with existing technologies, this store operation and equipment collaborative management system based on multimodal data fusion has the following advantages: I. This invention comprehensively acquires multimodal data from store operations by integrating logistics document interfaces, temperature and humidity sensors, weight sensors, and other data acquisition components. It cleans heterogeneous data, removes outliers, and unifies formats. Then, it extracts the spatial features of structured data and the temporal features of unstructured data. Through an attention mechanism, it completes data fusion and correlation. Combined with the built-in decision rule base, it generates instructions to extend the freshness of refrigeration equipment. The data fusion process achieves deep correlation of various types of scattered data. The decision rule base provides accurate basis for instruction generation. The equipment execution module adjusts the equipment operating status and provides feedback, forming a complete control process. This enables the operation of refrigeration equipment to accurately match the storage needs of goods, ensure the quality of goods storage, and achieve collaborative adaptation between data and equipment operation.

[0015] Second, this invention, through comprehensive collection and fusion processing of multimodal data, extracts key information such as shelf load-bearing capacity, merchandise inventory, and procurement cycle from collaborative analysis data. Relying on a decision rule base, it performs threshold comparison and analysis to generate an inventory layout optimization plan and predict inventory replenishment nodes. The inventory execution module pushes adjustment reminders to the work terminal, records the actual status of the inventory after adjustment, and feeds back to the fusion module, realizing dynamic adaptation of inventory layout and shelf load-bearing capacity. At the same time, it adjusts the operation mode of refrigeration equipment by combining procurement cycle and merchandise inventory data, so that inventory management, equipment operation, and supply chain information form deep collaboration, ensuring reasonable inventory configuration and equipment operation adaptation, and realizing orderly linkage of inventory, equipment, and supply chain links.

[0016] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0018] Figure 1 A flowchart of a store operation and equipment collaborative management system based on multimodal data fusion; Figure 2 This is a framework diagram of the fusion module in a store operation and equipment collaborative management system based on multimodal data fusion. Figure 3 This is a framework diagram of the decision-making module in a store operation and equipment collaborative management system based on multimodal data fusion. Detailed Implementation

[0019] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0020] Example 1: Data Acquisition Module: In the daily operation of a chain fresh food supermarket, the store mainly sells fresh produce such as fruits and vegetables, meat, and seafood, as well as daily necessities. Fresh produce accounts for 60% of the inventory, requiring high levels of refrigeration equipment preservation, shelf load-bearing balance, and inventory turnover efficiency. First, the system connects to the store's dedicated cold chain logistics system via a logistics document interface to read electronic logistics documents for fruits, vegetables, and meat delivered that day in real time. Information such as purchase order number, purchase cycle, arrival date, product name, shelf life, and quantity is extracted. Simultaneously, data is completed by associating the product name with the store's product information database. Temperature and humidity sensors are deployed in the fruit and vegetable, meat, and seafood refrigeration areas, collecting data every 5 minutes. The system collects ambient temperature and humidity data; weight sensors are installed under each shelf of the fruit and vegetable shelves and snack shelves to monitor the weight data corresponding to the inventory of goods on each shelf in real time; shelf load-bearing sensors are installed at the connection between the uprights and beams of all shelves to collect the load-bearing data of the entire shelf and each shelf in real time, with a focus on monitoring the load of the shelves storing heavy items such as watermelons and cantaloupes on the lower shelves; energy consumption sensors are deployed in the power supply circuits of the three refrigeration units to continuously record the real-time energy consumption of the equipment; equipment status monitoring sensors are integrated into the internal control unit of each refrigeration unit to synchronously collect data such as the start-stop status of the equipment and fault warning signals, and all collected data are uploaded to the fusion module in real time via wireless communication.

[0021] Fusion Module: After receiving multimodal data on store operations transmitted from the acquisition module, it initiates the processing flow. It first preprocesses the temperature and humidity data, merchandise inventory and weight data, shelf load-bearing data, equipment energy consumption data, equipment operating status data, and logistics document text data, such as... Figure 1 As shown, the process involves cleaning abnormal temperature and humidity values ​​caused by temporary sensor malfunctions, removing duplicate arrival quantity information from logistics documents, and standardizing heterogeneous data of different formats to eliminate format differences and noise interference. Subsequently, feature extraction is performed: the preprocessed structured sensor data is normalized to extract spatial features; the preprocessed unstructured logistics document text data is word-embedded and converted into vector form to extract temporal features; finally, an attention mechanism is used to fuse and associate the data. First, attention weights are determined based on the store's past year's records of near-expiry product handling, shelf load changes, and supply chain response data. Then, the spatial and temporal features are fused using the attention mechanism feature fusion formula to generate collaborative analysis data in a unified format. The attention mechanism feature fusion formula is as follows: ,in, These are the collaborative features after fusion, which are the core features of collaboratively analyzed data. For the first Spatial features, For the first One time-series feature, The total number of spatial features. The total number of time-series features. For the first Attention weights for each spatial feature relative to all temporal features For the first Attention weights of each temporal feature relative to all spatial features.

[0022] Decision Module: This module utilizes a built-in decision rule library, constructed based on one year of historical store operational data. It includes core rules for determining near-expiration thresholds for goods, standard storage temperature and humidity, shelf load-bearing safety thresholds, standard ranges of refrigeration equipment operating parameters, and inventory replenishment prediction thresholds. It extracts shelf-life data for meat products and temperature and humidity data for the meat refrigeration area from collaborative analysis data. The remaining shelf life of meat products is compared with the near-expiration thresholds in the decision rule library. When a batch of pork is found to have a remaining shelf life lower than the near-expiration threshold, the extension formula for refrigeration equipment shelf-life is used to determine the extension period. Then, based on the extension period and the standard storage temperature and humidity, the required temperature adjustment range for the refrigeration equipment is analyzed, generating a shelf-life extension instruction for the refrigeration equipment that includes temperature adjustment parameters. The shelf-life extension formula is as follows: ,in, To extend the shelf life, the temperature adjustment range of the refrigeration equipment is determined based on the extended shelf life. The longer the extended shelf life, the closer the temperature adjustment range is to the lower limit of the standard storage temperature and humidity for the product. This refers to the remaining shelf life of the product. This refers to the total shelf life of the product. This represents the difference between the current storage environment's temperature and humidity and the product's standard storage temperature and humidity. The basic freshness retention factor for goods is set based on the characteristics of the product category. The environmental temperature and humidity correction coefficient is determined by the deviation between the current storage environment temperature and humidity and the standard storage temperature and humidity for the goods. Simultaneously, the overall load-bearing data of each shelf and each shelf is extracted from the collaborative analysis data and compared with the shelf load-bearing safety threshold in the decision rule base. When the real-time load of the shelf storing watermelons on the lower shelf reaches 90% of the load-bearing safety threshold, the quantity of goods to be transferred is calculated using the shelf goods transfer quantity formula, combined with the real-time load-bearing data of other shelves, to generate an inventory layout optimization plan. The shelf goods transfer quantity formula is: ,in, The quantity of goods to be transferred. The load-bearing safety threshold of the shelf, The current load-bearing capacity of the shelf. The load balancing coefficient is determined based on statistical analysis of historical shelf load optimization data from stores. The unit weight of the goods to be transferred is obtained by breaking down the inventory weight data. In addition, the procurement cycle, arrival date and current inventory data of daily necessities are extracted from the collaborative analysis data. The current inventory of daily necessities is compared with the inventory replenishment prediction threshold in the decision rule base. Combined with the procurement cycle to predict the inventory replenishment node, the operating power and standby time of the corresponding refrigeration equipment are adjusted 24 hours before the replenishment node.

[0023] The device execution module establishes a connection with three refrigeration devices via a wireless LAN. After receiving the instruction from the decision module to extend the preservation time of the refrigeration devices, it lowers the set temperature of the refrigeration devices corresponding to the meat refrigeration zone and adjusts the operating power of the devices to maintain a stable low-temperature environment. During the parameter adjustment process, it monitors the operating status of the refrigeration devices in real time, records the adjusted set temperature, operating power, operating stability data, and energy consumption change data, and forms a time-series data set of device operating status according to the frequency consistent with the temperature and humidity sensors of the acquisition module. This data is then fed back to the fusion module for iterative optimization of attention weights.

[0024] Inventory Execution Module: After receiving the inventory layout optimization plan output by the decision-making module, it pushes adjustment reminder information to the mobile terminal of the stock clerk through the store's internal management platform, specifying that the name of the product to be adjusted is watermelon, the current storage location is the lower layer of shelf 1, and the target storage location is the middle layer of shelf 3. After the stock clerk completes the adjustment, he records the actual status of the inventory after the adjustment, including the completion status of the watermelon adjustment, the actual load data of each shelf of shelf 1 and shelf 3 after the adjustment, and synchronously feeds these status data and the refrigeration equipment operation status data back to the fusion module to form a complete closed-loop control.

[0025] In summary, in the scenario of chain fresh food supermarkets, the data acquisition module obtains multimodal data such as logistics documents, temperature and humidity, product inventory and weight, and shelf load-bearing capacity through various interfaces and sensors. The fusion module generates collaborative analysis data through preprocessing, feature extraction, and attention mechanism fusion. The decision-making module generates instructions to extend the shelf life of refrigerated equipment and optimize inventory layout based on the built-in decision rule library. At the same time, it predicts inventory replenishment nodes and adjusts the equipment operation mode. The equipment execution module and the inventory execution module adjust parameters and push reminders according to instructions, record status and feed back to the fusion module to form a complete closed loop, realizing precise coordination of fresh food storage, equipment operation and inventory configuration.

[0026] Example 2: Data Acquisition Module: In a community convenience store's peak replenishment scenario, located at the entrance of a residential area, the shopping peaks are during morning and evening commutes. The main products are snacks, daily necessities, and fresh food, such as sandwiches and rice balls. Replenishment frequency is high, requiring rapid response to inventory changes and equipment operational adaptability. Firstly, the system connects to the local supplier's logistics management system via a logistics document interface, synchronizing electronic logistics documents twice daily at 9:00 AM and 4:00 PM to extract information such as purchase order number, purchase cycle, arrival date, product name, product shelf life, and quantity received, with a focus on acquiring shelf life data for fresh food products. Temperature and humidity sensors are deployed only inside the fresh food refrigerated display case, monitoring every 3 minutes. The system collects temperature and humidity data of the storage environment to ensure a stable storage environment for fresh food; weight sensors are installed under each shelf of the snack and daily necessities shelves to monitor the weight of goods on each shelf in real time; shelf load-bearing sensors are installed at the connection between the shelf uprights and beams to monitor the load-bearing capacity of shelves storing heavier daily necessities such as beverages and laundry detergent; energy consumption sensors are deployed in the power supply circuit of the fresh food refrigerator to continuously record the real-time energy consumption of the equipment; and equipment status monitoring sensors are integrated into the internal control unit of the fresh food refrigerator to collect data such as the equipment's start-up and shutdown status and fault warning signals. All collected data is uploaded to the fusion module in real time via wireless communication.

[0027] The fusion module receives multimodal data from the acquisition module and initiates the processing flow. It first preprocesses temperature and humidity data, product inventory and weight data, shelf load-bearing data, equipment energy consumption data, equipment operating status data, and logistics document text data. It cleans the instantaneous fluctuations in temperature and humidity data from the refrigerated display case caused by opening and closing the door, removes incorrectly formatted delivery date information from logistics documents, and standardizes heterogeneous data to eliminate format differences and noise interference. Next, it performs feature extraction, normalizing the preprocessed structured sensor data and extracting spatial features, such as… Figure 2 As shown, the preprocessed unstructured logistics document text data is word-embedded and converted into vector form. Temporal features are extracted, and finally, an attention mechanism is used to complete data fusion and association. First, attention weights are determined based on the store's past six months of near-expiry product handling records, equipment load change data, and supply chain response data. Then, the spatial and temporal features are fused using the attention mechanism feature fusion formula to generate collaborative analysis data in a unified format. The attention mechanism feature fusion formula is as follows: .

[0028] Decision Module: This module utilizes a built-in decision rule library, constructed based on 18 months of historical store operational data. It includes core rules for determining near-expiration thresholds for goods, standard storage temperature and humidity, shelf load-bearing safety thresholds, standard ranges of refrigeration equipment operating parameters, and inventory replenishment prediction thresholds. It extracts shelf-life data for fresh foods such as sandwiches and rice balls, as well as temperature and humidity data for the refrigerated display cases, from collaborative analysis data. The remaining shelf life of the goods is compared to the near-expiration thresholds in the decision rule library. When a batch of rice balls is found to have a remaining shelf life lower than the near-expiration threshold, the extension formula for refrigeration equipment shelf-life is used to determine the extension period. Then, based on the extension period and the standard storage temperature and humidity, the required temperature adjustment range for the refrigeration equipment is analyzed, generating a shelf-life extension instruction for the refrigeration equipment that includes temperature adjustment parameters. The shelf-life extension formula is as follows: Simultaneously, the load-bearing data of each shelf is extracted from the collaborative analysis data and compared with the shelf load-bearing safety threshold in the decision rule base. When the real-time load of a shelf storing bottled beverages is detected to reach 90% of the load-bearing safety threshold, the real-time load-bearing data of other idle shelves is combined with the shelf merchandise transfer quantity formula to calculate the quantity of merchandise to be transferred, generating an inventory layout optimization plan, such as... Figure 3 As shown; the formula for the amount of goods transferred from the shelf is: In addition, the procurement cycle, arrival date and current inventory data of popular snack products during morning and evening peak hours are extracted from the collaborative analysis data. The current inventory is compared with the inventory replenishment prediction threshold in the decision rule base. Combined with the inventory replenishment node prediction based on the procurement cycle, it is determined to adjust the operating power and standby time of the fresh food refrigerator 1 hour before the morning peak and 1.5 hours before the evening peak.

[0029] The device execution module establishes a connection with the fresh food refrigerator via a wireless local area network. After receiving the instruction from the decision module to extend the preservation time of the refrigerator, it fine-tunes the set temperature of the fresh food refrigerator to ensure the preservation effect of near-expiry rice balls and sandwiches. During the parameter adjustment process, it monitors the operating status of the refrigerator in real time, records the adjusted set temperature, operating power, operating stability data and energy consumption change data, and forms a time-series data set of device operating status according to the frequency consistent with the temperature and humidity sensors of the acquisition module, and feeds it back to the fusion module.

[0030] Inventory Execution Module: After receiving the inventory layout optimization plan output by the decision-making module, it pushes adjustment reminder information to the store staff's mobile terminals through the store's internal management platform, specifying that the product to be adjusted is bottled cola, the current storage location is the lower shelf of shelf 2, and the target storage location is the lower shelf of shelf 4. After the store staff completes the transfer of goods before the peak period, they record the actual status of the inventory after the adjustment, including the completion status of the bottled cola adjustment and the actual load data of shelves 2 and 4 after the adjustment. This status data is synchronized with the fresh food refrigerated display case operation status data and fed back to the fusion module to ensure smooth store operation during peak periods.

[0031] In summary, the data collection module synchronizes logistics document data on a regular basis, and collects targeted data on temperature and humidity in the fresh food refrigeration area, weight of goods on shelves, and equipment operation. The fusion module processes heterogeneous data and completes feature fusion through an attention mechanism. The decision-making module generates instructions to extend the freshness of fresh food refrigeration equipment and optimizes the inventory layout for transferring goods on shelves based on collaborative analysis data and a decision rule base. It also predicts the inventory replenishment nodes for best-selling products and adjusts the equipment operation mode. The equipment execution module adjusts the parameters of the refrigeration equipment, and the inventory execution module pushes reminders to the work terminal and records the adjustment status. Both modules provide synchronous feedback data to ensure efficient and smooth store operations during peak hours.

[0032] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A store operation and equipment collaborative management system based on multimodal data fusion, characterized in that: The system includes: Data Acquisition Module: Integrates logistics document interface, temperature and humidity sensor, weight sensor, shelf load sensor, energy consumption sensor and equipment status monitoring sensor to acquire multimodal data of store operation; Fusion module: It preprocesses the acquired multimodal data, performs feature extraction operations on structured and unstructured data respectively, and completes data fusion and association through attention mechanism to generate collaborative analysis data in a unified format; Decision module: Built-in decision rule library, based on collaborative analysis data and decision rule library, dynamically generates instructions to extend the freshness preservation time of refrigeration equipment and inventory layout optimization plan, and predicts inventory replenishment nodes to adjust the operation mode of refrigeration equipment; Equipment execution module: Based on the instruction to extend the preservation time of the refrigeration equipment, the operating parameters of the refrigeration equipment are adjusted in real time, and the operating status of the refrigeration equipment after adjustment is recorded and fed back to the fusion module; Inventory Execution Module: Based on the inventory layout optimization scheme, it sends adjustment reminder information to the work terminal, records the actual status of the inventory after adjustment, and feeds back to the fusion module.

2. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 1, characterized in that, The data acquisition module connects to the logistics system via a logistics document interface to obtain unstructured text information from the logistics documents, including purchase order number, purchase cycle, arrival date, product name, product shelf life, and quantity received. Temperature and humidity sensors are deployed in the product storage area to collect temperature and humidity data of the storage environment. Weight sensors are installed under the shelf shelves to monitor the weight data of the corresponding products on each shelf. Shelf load-bearing sensors are installed at the connection between the shelf uprights and beams to collect real-time load-bearing data of the entire shelf and each shelf. Energy consumption sensors are deployed in the power supply circuit of the refrigeration equipment to record real-time operating energy consumption. Equipment status monitoring sensors are integrated into the internal control unit of the refrigeration equipment to collect equipment operating status data such as the start / stop status and fault warning signals of the refrigeration equipment.

3. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 1, characterized in that, In the fusion module, data cleaning, outlier removal, and format unification preprocessing operations are performed on temperature and humidity data, commodity inventory and weight data, shelf load-bearing data, equipment energy consumption data, equipment operating status data, and logistics document text data in the multimodal data. This eliminates format differences and noise interference from heterogeneous data. Then, feature extraction is performed. The preprocessed structured sensor data is normalized and spatial features are extracted. The preprocessed unstructured text data is word-embedded and converted into vector form to extract temporal features. Finally, a fusion and association operation is performed. The extracted spatial and temporal features are associated and fused through an attention mechanism feature fusion formula to generate collaborative analysis data in a unified format.

4. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 3, characterized in that, In the fusion module, when the extracted spatial features and temporal features are correlated and fused using the attention mechanism feature fusion formula, attention weights are first assigned. The attention weights are determined based on the historical operation data statistics of the store. The historical operation data includes past records of near-expiry product handling, equipment load change data, and supply chain response data. Then, the fusion of spatial features and temporal features is completed using the attention mechanism feature fusion formula.

5. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 4, characterized in that, In the fusion module, the formula for fusion of attention mechanism features is: ,in, For the synergistic features after fusion, For the first Spatial features, For the first One time-series feature, The total number of spatial features. The total number of time-series features. For the first Attention weights for each spatial feature relative to all temporal features For the first Attention weights of each temporal feature relative to all spatial features.

6. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 1, characterized in that, In the decision-making module, the decision rule base is built based on historical store operation data, including core rules such as product near-expiration judgment threshold, standard storage temperature and humidity for products, shelf load-bearing safety threshold, standard range of refrigeration equipment operating parameters, and inventory replenishment prediction threshold. It extracts product shelf-life data and storage environment temperature and humidity data from collaborative analysis data, compares the remaining shelf-life of products with the product near-expiration judgment threshold in the decision rule base, and if it is lower than the near-expiration judgment threshold, firstly, it determines the refrigeration equipment's shelf-life extension time using a formula for extending shelf-life, and then, based on the shelf-life extension time and the standard storage temperature and humidity of the products, analyzes the temperature range that the refrigeration equipment needs to adjust, generating a refrigeration equipment shelf-life extension instruction that includes temperature adjustment parameters. It also extracts the overall shelf-life and the load-bearing data of each shelf from the collaborative analysis data and compares it with the shelf load-bearing safety threshold in the decision rule base. When it reaches 90% of the load-bearing safety threshold, it combines real-time load-bearing data from other shelves and determines the quantity of products to be transferred using a shelf-life transfer quantity formula, generating an inventory layout optimization plan.

7. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 6, characterized in that, In the decision-making module, the formula for extending the shelf life is: ,in, To extend the shelf life, the temperature adjustment range of the refrigeration equipment is determined based on the extended shelf life. The longer the extended shelf life, the closer the temperature adjustment range is to the lower limit of the standard storage temperature and humidity for the product. This refers to the remaining shelf life of the product. This refers to the total shelf life of the product. This represents the difference between the current storage environment's temperature and humidity and the product's standard storage temperature and humidity. The basic preservation coefficient of the product, This is the correction factor for ambient temperature and humidity.

8. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 6, characterized in that, In the decision-making module, the formula for the amount of goods transferred from the shelf is: ,in, The quantity of goods to be transferred. The load-bearing safety threshold of the shelf, The current load-bearing capacity of the shelf. This is the load balancing coefficient. This refers to the unit weight of the goods to be transferred.

9. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 1, characterized in that, In the decision-making module, the procurement cycle, delivery date and current product inventory data are extracted from the collaborative analysis data. The current product inventory is compared with the inventory replenishment prediction threshold in the decision rule base. The inventory replenishment node is predicted in combination with the procurement cycle, and the operating power and standby time of the refrigeration equipment are adjusted before the replenishment node.

10. The store operation and equipment collaborative management system based on multimodal data fusion according to claim 1, characterized in that, In the inventory execution module, the adjustment reminder information includes the name of the product to be adjusted, the current storage location, and the target storage location; the recorded actual status after inventory adjustment includes the completion status of product adjustment and the actual load data of each shelf after adjustment.

Images