A commercial circulation supply chain optimization method based on multimodal data fusion

Abstract

The present application discloses a commercial circulation supply chain optimization method based on multimodal data fusion, which includes the following steps: Step 1, multimodal data collection: through Internet of Things devices, enterprise ERP systems, logistics tracking systems and third-party platforms, multimodal data of the entire supply chain are collected in real time. Multimodal data includes text data, image data, voice data, time series data and spatial data. When this solution is implemented, by integrating multi-source heterogeneous data such as the Internet of Things and ERP, the problem of information islands in the traditional supply chain is effectively broken; the position encoding and cross-modal attention mechanism are introduced into the multimodal fusion model to significantly improve the accuracy of data alignment. For example, when the "urgent order" in the voice communication record is associated with the corresponding order image, the accuracy of weight allocation is improved; the application of blockchain technology ensures the transparency and traceability of supply chain decisions.

Description

Technical Field

[0001] The present application relates to the field of computer data processing technology, and in particular to a commercial circulation supply chain optimization method based on multimodal data fusion. Background Art

[0002] The supply chain is a critical link throughout the entire commercial process, and its operational efficiency and quality directly impact a company's competitiveness and market performance. However, the current supply chain system presents numerous issues that need to be addressed.

[0003] On the one hand, core links in the supply chain, such as procurement, warehousing, and logistics, operate independently, resulting in highly fragmented data. The data generated by each link resembles isolated "information islands," lacking effective interconnection and deep integration, and cross-modal data fusion capabilities are extremely weak. This situation makes it difficult for companies to quickly and accurately obtain comprehensive and integrated information when faced with complex and volatile market environments and business needs, leading to delayed decision-making. Companies are unable to promptly adjust procurement plans, optimize warehousing layouts, or plan efficient logistics routes based on market dynamics, resulting in missed business opportunities and even financial losses due to delayed decision-making.

[0004] On the other hand, traditional supply chain management methods have serious limitations in data processing, relying too heavily on single sensor data or relying solely on text-based analysis. In today's digital age, data forms are becoming increasingly diverse, encompassing not only traditional structured data but also large amounts of unstructured data such as images and voice. However, traditional methods struggle to process this unstructured data, making it difficult to effectively integrate and utilize it. For example, in order processing, the error rate for handwritten order recognition remains high. This not only increases the workload for manual review and reduces business processing efficiency, but can also lead to order information deviations due to recognition errors, impacting customer satisfaction and the normal operation of the business. Summary of the Invention

[0005] To this end, the present application provides a commercial circulation supply chain optimization method based on multimodal data fusion to solve the problems of highly dispersed data and serious limitations in data processing in the existing technology.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A commercial circulation supply chain optimization method based on multimodal data fusion includes the following steps:

[0008] Step 1: Multimodal data collection: Through IoT devices, enterprise ERP systems, logistics tracking systems, and third-party platforms, multimodal data from the entire supply chain is collected in real time. Multimodal data includes text data, image data, voice data, time series data, and spatial data. Text data includes order information and contract terms. Image data includes product images and logistics documents. Voice data includes customer call records. Time series data includes inventory changes and transportation trajectories. Spatial data includes warehouse locations and delivery routes.

[0009] Step 2: Data preprocessing and feature extraction: Multimodal data is cleaned, standardized, and denoised. A Transformer-based multimodal encoder is used to extract features from text, image, and speech data to generate high-dimensional semantic vectors. Dynamic features are extracted from time series data using an LSTM network, and spatial data is converted into coordinate vectors through geocoding.

[0010] Step 3: Multimodal data fusion optimization: Build a position-sensitive optimized multimodal fusion model. Input the feature vectors extracted in Step 2 into the fusion model. Using a cross-modal attention mechanism, calculate the association weights for text-image, text-speech, and time-series-spatial data to generate a unified fused feature matrix. The fusion model combines the transition probability matrix to predict supply chain status and dynamically adjusts the weight distribution based on real-time data.

[0011] Step 4: Intelligent Supply Chain Decision-Making: Based on a unified feature matrix, a deep reinforcement learning algorithm is used to generate a supply chain optimization strategy. The supply chain optimization strategy includes dynamic inventory allocation, route planning, supplier matching, and risk warning.

[0012] Step 5: Feedback and iteration: Use blockchain technology to record the results of supply chain decision execution, feed the actual benefit data back to the fusion model, and use the differential evolution algorithm to optimize the model parameters to achieve closed-loop iteration.

[0013] Preferably, the data preprocessing in step 2 includes:

[0014] Use layout analysis models to locate key areas of image data to obtain product names and quantities;

[0015] The voice data is mapped through the dialect recognition module and product library to eliminate homophone ambiguity;

[0016] Entity recognition technology is used to extract supply chain entities from text data. Supply chain entities include suppliers, SKUs, and delivery times.

[0017] Preferably, the position-sensitively optimized multimodal fusion model in step 3 specifically includes a cross-modal alignment module, a cultural center model, and a transition probability matrix;

[0018] The cross-modal alignment module is based on position encoding and is used to eliminate spatial bias between different data sources.

[0019] The Wenxin model has a layered processing architecture for efficient semantic parsing;

[0020] The transition probability matrix is ​​used to predict future risk probabilities by combining historical supply chain status data.

[0021] Preferably, the deep reinforcement learning algorithm in step 4 adopts a proximal policy optimization framework, which has a reward function, and the reward function is:

[0022] R = a × order fulfillment rate + b × inventory turnover rate - c × logistics delay penalty;

[0023] Among them, a, b, and c are weight coefficients, R is the reward value, and a, b, and c are dynamically adjusted based on historical data;

[0024] Evaluate the circulation efficiency of goods based on the R value.

[0025] Preferably, the blockchain technology described in step 5 is specifically:

[0026] Build a consortium chain to store supply chain decision logs and execution results;

[0027] The data feedback and model iteration process is automatically triggered through smart contracts.

[0028] Preferably, the method further includes a dynamic path planning step:

[0029] Based on real-time traffic data and delivery needs, route selection is optimized through a reward and penalty mechanism. If congestion or abnormal weather conditions are predicted on the route, the route will be replanned.

[0030] Preferably, the method for constructing the transition probability matrix in step 3 includes:

[0031] Define the supply chain status set as {S1: normal operation, S2: inventory shortage, S3: logistics delay};

[0032] Calculate state transition probability based on historical data.

[0033] Preferably, the step 1 further includes an input module, which supports uploading data by voice input and picture uploading.

[0034] Compared with the prior art, this application has at least the following beneficial effects:

[0035] 1. When implementing this solution, by integrating heterogeneous data from multiple sources such as the Internet of Things and ERP, the problem of information silos in the traditional supply chain can be effectively broken;

[0036] 2. The introduction of position encoding and cross-modal attention mechanisms in the multimodal fusion model significantly improves data alignment accuracy. For example, when associating "urgent order" in voice communication records with corresponding order images, the accuracy of weight allocation is improved;

[0037] 3. The application of blockchain technology ensures the transparency and traceability of supply chain decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more intuitively illustrate the prior art and the present application, exemplary drawings are provided below. It should be understood that the specific shapes and structures shown in the drawings should not generally be considered as limiting conditions for implementing the present application. For example, based on the technical concepts disclosed in this application and the exemplary drawings, those skilled in the art are capable of easily making routine adjustments or further optimizations to the addition / reduction / attribution division of certain units (components), the specific shapes, positional relationships, connection methods, and dimensional ratios.

[0039] Figure 1 This is a flowchart of a commercial circulation supply chain optimization method based on multimodal data fusion in this application. DETAILED DESCRIPTION

[0040] The present application will be further described below in detail through specific embodiments in conjunction with the accompanying drawings.

[0041] like Figure 1 As shown, a commercial circulation supply chain optimization method based on multimodal data fusion includes the following steps:

[0042] Step 1: Multimodal data collection: Through IoT devices, enterprise ERP systems, logistics tracking systems, and third-party platforms, multimodal data from the entire supply chain is collected in real time. Multimodal data includes text data, image data, voice data, time series data, and spatial data. Text data includes order information and contract terms. Image data includes product images and logistics documents. Voice data includes customer call records. Time series data includes inventory changes and transportation trajectories. Spatial data includes warehouse locations and delivery routes.

[0043] Step 2: Data preprocessing and feature extraction: Multimodal data is cleaned, standardized, and denoised. A Transformer-based multimodal encoder is used to extract features from text, image, and speech data to generate high-dimensional semantic vectors. Dynamic features are extracted from time series data using an LSTM network, and spatial data is converted into coordinate vectors through geocoding. A hybrid model combining Transformer and LSTM is used to capture semantic associations between text and images (such as matching contract terms with product images) and model long-term dependencies on inventory fluctuations.

[0044] Step 3: Multimodal data fusion optimization: Build a position-sensitive optimized multimodal fusion model. Input the feature vectors extracted in Step 2 into the fusion model. Using a cross-modal attention mechanism, calculate the association weights for text-image, text-speech, and time-series-spatial data to generate a unified fused feature matrix. The fusion model combines the transition probability matrix to predict supply chain status and dynamically adjusts the weight distribution based on real-time data.

[0045] Step 4: Intelligent Supply Chain Decision-Making: Based on a unified feature matrix, a deep reinforcement learning algorithm is used to generate a supply chain optimization strategy. The supply chain optimization strategy includes dynamic inventory allocation, route planning, supplier matching, and risk warning.

[0046] Step 5: Feedback and iteration: Use blockchain technology to record the results of supply chain decision execution, feed the actual benefit data back to the fusion model, and use the differential evolution algorithm to optimize the model parameters to achieve closed-loop iteration.

[0047] This invention achieves end-to-end intelligent optimization by building a multimodal data fusion and decision-making system covering the entire supply chain. During the data collection phase, it integrates heterogeneous data from multiple sources, such as the Internet of Things and ERP, effectively breaking down the information silos in traditional supply chains. For example, time-series data acquired through logistics tracking systems can reflect changes in cargo location in real time. When combined with warehouse space data, this can predict sorting pressure in advance.

[0048] The introduction of position encoding and cross-modal attention mechanisms in the multimodal fusion model significantly improves data alignment accuracy. For example, when associating "urgent order" in voice communication records with corresponding order images, the accuracy of weight allocation is improved.

[0049] Dynamic strategies generated through reinforcement learning (such as inventory allocation under sudden demand) can improve order fulfillment rates, while the blockchain-based feedback loop ensures continuous optimization of the model and avoids decision failures caused by data drift.

[0050] The overall solution compresses the supply chain response speed from hours to minutes, greatly reducing the overall operating costs and speeding up the response speed in the trade process.

[0051] The data preprocessing in step 2 includes:

[0052] Use layout analysis models to locate key areas of image data to obtain product names and quantities;

[0053] The voice data is mapped through the dialect recognition module and product library to eliminate homophone ambiguity;

[0054] Entity recognition technology is used to extract supply chain entities from text data. Supply chain entities include suppliers, SKUs, and delivery times.

[0055] The above technical solution proposes one optimization scheme for the preprocessing of multimodal data, significantly improving the reliability of subsequent analysis. Image processing utilizes layout analysis models (such as PicoDet_layout_1x) to accurately locate key fields (such as barcodes and receipt dates) in logistics documents, avoiding the recognition errors caused by traditional optical character recognition (OCR) due to document wrinkles. Voice data is mapped to a product library through dialect recognition to resolve ambiguity in the same phrase across different scenarios, such as the ambiguity between "apple" in fresh produce and electronics, thereby improving the accuracy of semantic analysis. This preprocessing approach significantly reduces the interference of noisy data on the fusion model, increasing the confidence level of the input data for subsequent decision modules.

[0056] The position-sensitively optimized multimodal fusion model described in step 3 specifically includes a cross-modal alignment module, a cultural model, and a transition probability matrix;

[0057] The cross-modal alignment module is based on position encoding and is used to eliminate spatial bias between different data sources.

[0058] The Wenxin model has a layered processing architecture for efficient semantic parsing;

[0059] The transition probability matrix is ​​used to predict future risk probabilities by combining historical supply chain status data.

[0060] For example, when analyzing a customer's voice complaint about "damaged goods", the mechanism will prioritize associating the logistics image data of the corresponding batch (such as photos of damaged outer packaging) rather than irrelevant inventory text records, thereby improving the association accuracy.

[0061] The deep reinforcement learning algorithm described in step 4 uses a proximal policy optimization framework with a reward function, which is:

[0062] R = a × order fulfillment rate + b × inventory turnover rate - c × logistics delay penalty;

[0063] Among them, a, b, and c are weight coefficients, R is the reward value, and a, b, and c are dynamically adjusted based on historical data;

[0064] Evaluate the circulation efficiency of goods based on the R value.

[0065] As one of the implementation scenarios of the above solution, when a=0.5, b=0.3, and c=0.2, resources can be automatically allocated to ensure the supply of popular products in the Double Eleven promotion scenario, greatly improving the order fulfillment rate. At the same time, the logistics delay rate can be controlled at a low level through the logistics delay penalty mechanism.

[0066] The blockchain technology described in step 5 is specifically:

[0067] Build a consortium chain to store supply chain decision logs and execution results;

[0068] The data feedback and model iteration process is automatically triggered through smart contracts.

[0069] The application of blockchain technology ensures transparency and traceability in supply chain decision-making. When using a consortium blockchain to store data, each node (supplier, logistics provider, retailer) can share decision logs under permission control, resolving the trust issues of traditional centralized systems and ensuring traceability in dispute resolution.

[0070] For the smart contract's automatic execution feedback process: when the actual delivery time exceeds the predicted value by 10%, the model parameter adjustment instruction is immediately triggered, and the iteration cycle is shortened from weeks to hours.

[0071] It also includes a dynamic path planning step:

[0072] Based on real-time traffic data and delivery needs, route selection is optimized through a reward and penalty mechanism. If congestion or abnormal weather conditions are predicted on the route, the route will be replanned.

[0073] Dynamic route planning utilizes real-time data to achieve flexible scheduling. When the traffic monitoring API indicates congestion on a particular road section, the system automatically applies a penalty factor (e.g., original route score multiplied by 0.6) and recalculates the optimal route. Combined with weather API data, low-lying roads can be avoided in advance during heavy rain warnings.

[0074] The method for constructing the transition probability matrix in step 3 includes:

[0075] Define the supply chain status set as {S1: normal operation, S2: inventory shortage, S3: logistics delay};

[0076] Calculate state transition probability based on historical data.

[0077] After defining the state set {S1, S2, S3}, the transition probability is calculated based on historical data. When real-time data monitors that the inventory level is approaching the safety threshold (for example, when the inventory level is 10%), the model predicts that the probability of entering the S2 state within the next 72 hours will exceed 30%, immediately triggering an early warning and automatically generating a replenishment strategy.

[0078] Step 1 also includes an input module, which supports uploading data through voice input and image upload. The customized design for commercial and wholesale scenarios greatly improves practical efficiency. Voice input supports real-time translation of dialects (such as the Cantonese "urgent three boxes of goods" is automatically converted into a standard order). Combined with automatic recognition of image uploads (such as scanning of drug batch numbers), the ordering process is compressed.

[0079] The technical features of the above embodiments can be combined arbitrarily (as long as there is no contradiction in the combination of these technical features). In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described; these embodiments that are not explicitly written should also be considered to be within the scope of this specification.

Claims

1. A commercial circulation supply chain optimization method based on multimodal data fusion, characterized by: The following steps are included: Step 1: Multimodal data collection: Through IoT devices, enterprise ERP systems, logistics tracking systems, and third-party platforms, multimodal data from the entire supply chain is collected in real time. Multimodal data includes text data, image data, voice data, time series data, and spatial data. Text data includes order information and contract terms. Image data includes product images and logistics documents. Voice data includes customer call records. Time series data includes inventory changes and transportation trajectories. Spatial data includes warehouse locations and delivery routes. Step 2: Data preprocessing and feature extraction: Multimodal data is cleaned, standardized, and denoised. A Transformer-based multimodal encoder is used to extract features from text, image, and speech data to generate high-dimensional semantic vectors. Dynamic features are extracted from time series data using an LSTM network, and spatial data is converted into coordinate vectors through geocoding. Step 3: Multimodal data fusion optimization: Build a position-sensitive optimized multimodal fusion model. Input the feature vectors extracted in Step 2 into the fusion model. Using a cross-modal attention mechanism, calculate the association weights for text-image, text-speech, and time-series-spatial data to generate a unified fused feature matrix. The fusion model combines the transition probability matrix to predict supply chain status and dynamically adjusts the weight distribution based on real-time data. The position-sensitive optimized multimodal fusion model includes a cross-modal alignment module, which is based on position encoding and is used to eliminate spatial deviations between different data sources; Step 4: Intelligent Supply Chain Decision-Making: Based on a unified feature matrix, a deep reinforcement learning algorithm is used to generate a supply chain optimization strategy. The supply chain optimization strategy includes dynamic inventory allocation, route planning, supplier matching, and risk warning. The deep reinforcement learning algorithm described in step 4 uses a proximal policy optimization framework with a reward function, which is: R = a × order fulfillment rate + b × inventory turnover rate - c × logistics delay penalty; Among them, a, b, and c are weight coefficients, R is the reward value, and a, b, and c are dynamically adjusted based on historical data; Evaluate the circulation efficiency of goods based on the R value; Step 5: Feedback and iteration: Use blockchain technology to record the results of supply chain decision execution, feed the actual benefit data back to the fusion model, and use the differential evolution algorithm to optimize the model parameters to achieve closed-loop iteration.

2. The commercial circulation supply chain optimization method based on multimodal data fusion according to claim 1 is characterized in that: The data preprocessing in step 2 includes: Use layout analysis models to locate key areas of image data to obtain product names and quantities; The voice data is mapped through the dialect recognition module and product library to eliminate homophone ambiguity; Entity recognition technology is used to extract supply chain entities from text data. Supply chain entities include suppliers, SKUs, and delivery times.

3. The commercial circulation supply chain optimization method based on multimodal data fusion according to claim 1 is characterized in that: The multimodal fusion model described in step 3 specifically includes the Wenxin model and the transition probability matrix; The Wenxin model has a layered processing architecture for efficient semantic parsing; The transition probability matrix is ​​used to predict future risk probabilities by combining historical supply chain status data.

4. The commercial circulation supply chain optimization method based on multimodal data fusion according to claim 1 is characterized in that: The blockchain technology described in step 5 is specifically: Build a consortium chain to store supply chain decision logs and execution results; The data feedback and model iteration process is automatically triggered through smart contracts.

5. The commercial circulation supply chain optimization method based on multimodal data fusion according to claim 1 is characterized in that: It also includes a dynamic path planning step: Based on real-time traffic data and delivery needs, route selection is optimized through a reward and penalty mechanism. If congestion or abnormal weather conditions are predicted on the route, the route will be replanned.

6. The commercial circulation supply chain optimization method based on multimodal data fusion according to claim 1 is characterized in that: The method for constructing the transition probability matrix in step 3 includes: Define the supply chain status set as {S1: normal operation, S2: inventory shortage, S3: logistics delay}; Calculate state transition probability based on historical data.

7. The commercial circulation supply chain optimization method based on multimodal data fusion according to claim 1 is characterized in that: The step 1 also includes an input module, which supports uploading data through voice input and picture uploading.

Images