Commercial and trade circulation supply chain optimization method based on multi-modal data fusion

Abstract

The invention discloses a commerce and trade circulation supply chain optimization method based on multi-modal data fusion, and the method comprises the following steps: 1, multi-modal data collection: collecting the multi-modal data of the whole supply chain in real time through an Internet of Things device, an enterprise ERP system, a logistics tracking system and a third-party platform, the multi-modal data comprises text data, image data, voice data, time sequence data and spatial data. When the scheme is implemented, multi-source heterogeneous data of the Internet of Things, ERP and the like are integrated, so that the problem of an information island in a traditional supply chain is effectively solved; position coding and a cross-modal attention mechanism are introduced into the multi-modal fusion model, so that the data alignment precision is remarkably improved, for example, when'emergency order adding 'in a voice communication record is associated with a corresponding order image, the weight distribution accuracy is improved; the application of the block chain technology ensures the transparency and traceability of the supply chain decision.

Description

Technical Field

[0001] This application relates to the technical field of computer data processing, and particularly to an optimization method for the commercial circulation supply chain based on multi-modal data fusion. Background Art

[0002] In the process of promoting commercial activities, the supply chain, as a key link throughout the process, its operating efficiency and quality directly affect the competitiveness and market performance of enterprises. However, many problems in the current supply chain system urgently need to be solved.

[0003] On the one hand, the core links of the supply chain, such as procurement, warehousing, and logistics, act independently, and the data is in a highly dispersed state. The data generated in each link is like "information islands", lacking effective interconnection and deep integration with each other, and the cross-modal data fusion ability is extremely weak. This situation makes it difficult for enterprises to quickly and accurately obtain comprehensive and integrated information when facing complex and changing market environments and business requirements, resulting in a lag in the decision-making process. Enterprises cannot adjust procurement plans, optimize warehousing layouts, or plan efficient logistics routes in a timely manner according to market dynamics, missing many business opportunities, and may even suffer economic losses due to delayed decisions.

[0004] On the other hand, traditional supply chain management methods have serious limitations in data processing, relying too much on single-sensor data or only text analysis. In today's digital age, data forms are becoming increasingly diverse. In addition to traditional structured data, there is also a large amount of unstructured data such as images and voices. However, traditional methods are unable to handle these unstructured data effectively and are difficult to integrate and utilize. For example, in the order processing link, the recognition error rate of handwritten orders remains high, which not only increases the workload of manual review, reduces the business processing efficiency, but also may cause order information deviation due to recognition errors, affecting customer satisfaction and the normal operation of enterprises. Summary of the Invention

[0005] Therefore, this application provides an optimization method for the commercial circulation supply chain based on multi-modal data fusion to solve the problems of highly dispersed data and serious limitations in data processing existing in the prior art.

[0006] To achieve the above object, this application provides the following technical solutions:

[0007] An optimization method for the commercial circulation supply chain based on multi-modal data fusion includes the following steps,

[0008] Step 1, Multimodal Data Collection: Through Internet of Things devices, enterprise ERP systems, logistics tracking systems, and third-party platforms, real-time collection of multimodal data across the entire supply chain. Multimodal data includes text data, image data, voice data, time-series data, and spatial data. Text data includes order information and contract terms. Image data includes product pictures and logistics documents. Voice data includes call communication records with customers. Time-series data includes inventory changes and transportation trajectories. Spatial data includes warehouse locations and delivery routes;

[0009] Step 2, Data Preprocessing and Feature Extraction: Clean, standardize, and denoise the multimodal data. Use a Transformer-based multimodal encoder to extract features from text, image, and voice data, generating high-dimensional semantic vectors. Time-series data extracts dynamic features through an LSTM network, and spatial data is converted into coordinate vectors through geocoding;

[0010] Step 3, Multimodal Data Fusion and Optimization: Build a location-sensitive optimized multimodal fusion model. Input the feature vectors extracted in Step 2 into the fusion model. Calculate the correlation weights of text-image, text-voice, and time-series-spatial data through a cross-modal attention mechanism, generating a unified feature matrix after fusion. The fusion model combines a transition probability matrix to predict the supply chain status and dynamically adjusts the weight allocation according to real-time data;

[0011] Step 4, Supply Chain Intelligent Decision-making: Based on the unified feature matrix, use a deep reinforcement learning algorithm to generate supply chain optimization strategies. Supply chain optimization strategies include dynamic inventory allocation, route planning, supplier matching, and risk warning;

[0012] Step 5, Feedback and Iteration: Record the execution results of supply chain decisions through blockchain technology, feedback the actual benefit data to the fusion model, and use the differential evolution algorithm to optimize the model parameters to achieve closed-loop iteration.

[0013] Preferably, the data preprocessing in Step 2 includes:

[0014] For image data, use a layout analysis model to locate key regions to obtain product names and quantities;

[0015] For voice data, eliminate homophone ambiguities through a dialect recognition module and mapping with a product library;

[0016] For text data, use entity recognition technology to extract supply chain entities. Supply chain entities include suppliers, SKUs, and delivery times.

[0017] Preferably, the location-sensitive optimized multimodal fusion model in Step 3 specifically includes a cross-modal alignment module, a Wenxin Big Model, and a transition probability matrix;

[0018] The cross-modal alignment module is based on positional encoding and is used to eliminate the spatial deviation of different data sources;

[0019] The ERNIE model has a hierarchical processing architecture for efficient semantic parsing;

[0020] The transition probability matrix is used to predict future risk probabilities by combining historical supply chain status data.

[0021] Preferably, in step four, the deep reinforcement learning algorithm adopts the Proximal Policy Optimization framework, which has a reward function, and the reward function is:

[0022] R = a × order fulfillment rate + b × inventory turnover rate - c × logistics delay penalty;

[0023] where a, b, and c are weight coefficients, R is the reward value, and a, b, and c are dynamically adjusted through historical data;

[0024] The flow efficiency of the goods is evaluated according to the R value.

[0025] Preferably, the blockchain technology described in step five is specifically:

[0026] A consortium chain is constructed to store the supply chain decision logs and execution results;

[0027] The data feedback and model iteration process are automatically triggered through smart contracts.

[0028] Preferably, it further includes a dynamic path planning step:

[0029] Based on real-time traffic data and distribution requirements, the path selection is optimized through a reward and punishment mechanism. If congestion or abnormal weather exists in the predicted path, the path is re-planned.

[0030] Preferably, the construction method of the transition probability matrix described in step three includes:

[0031] Define the supply chain state set as {S1: normal operation, S2: inventory shortage, S3: logistics delay};

[0032] Calculate the state transition probability according to historical data.

[0033] Preferably, the input module is further included in step one, and the input module supports uploading data by voice input and picture upload.

[0034] Compared with the prior art, the present application has at least the following beneficial effects:

[0035] 1. When the present solution is implemented, by integrating multi-source heterogeneous data such as the Internet of Things and ERP, the problem of information islands in the traditional supply chain is effectively solved;

[0036] 2. In the multi-modal fusion model, the introduction of positional encoding and cross-modal attention mechanism significantly improves the data alignment accuracy. For example, when associating "urgent additional order" in the voice communication record with the corresponding order image, the accuracy of weight assignment is improved.

[0037] 3. The application of blockchain technology ensures the transparency and traceability of supply chain decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more intuitively illustrate the prior art and the present application, exemplary drawings are given below. It should be understood that the specific shapes and structures shown in the drawings generally should not be regarded as limiting conditions when implementing the present application. For example, those skilled in the art are capable of making routine adjustments or further optimizations to the addition / deletion / attribution division of certain units (components), specific shapes, positional relationships, connection methods, dimensional proportional relationships, etc. based on the technical concept disclosed in the present application and the exemplary drawings.

[0039] Figure 1 FIG. is a flowchart of an optimization method for a business circulation supply chain based on multi-modal data fusion according to the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] The present application will be further described in detail below with reference to the drawings through specific embodiments.

[0041] As Figure 1 shown, an optimization method for a business circulation supply chain based on multi-modal data fusion includes the following steps.

[0042] Step 1, multi-modal data collection: Through Internet of Things devices, enterprise ERP systems, logistics tracking systems, and third-party platforms, multi-modal data of the entire supply chain is collected in real time. The multi-modal data includes text data, image data, voice data, time-series data, and spatial data. The text data includes order information and contract terms. The image data includes product pictures and logistics documents. The voice data includes call communication records with customers. The time-series data includes inventory changes and transportation trajectories. The spatial data includes warehouse locations and delivery routes.

[0043] Step 2, data preprocessing and feature extraction: The multi-modal data is cleaned, standardized, and denoised. A multi-modal encoder based on Transformer is used to extract features from text, image, and voice data to generate high-dimensional semantic vectors. The time-series data extracts dynamic features through an LSTM network. The spatial data is converted into coordinate vectors through geocoding. A hybrid model combining Transformer and LSTM can capture the semantic associations of text and images (such as the matching verification of contract terms and product pictures) and can also model the long-term dependence relationship of inventory fluctuations.

[0044] Step 3, Multimodal Data Fusion Optimization: Construct a location-sensitive optimized multimodal fusion model. Input the feature vectors extracted in Step 2 into the fusion model. Calculate the correlation weights of text-image, text-speech, and time-series-spatial data through a cross-modal attention mechanism to generate a unified feature matrix after fusion. The fusion model combines the transition probability matrix to predict the supply chain status and dynamically adjusts the weight allocation according to real-time data;

[0045] Step 4, Supply Chain Intelligent Decision-making: Based on the unified feature matrix, adopt a deep reinforcement learning algorithm to generate supply chain optimization strategies, which include dynamic inventory allocation, route planning, supplier matching, and risk warning;

[0046] Step 5, Feedback and Iteration: Record the execution results of supply chain decisions through blockchain technology, feedback the actual benefit data to the fusion model, and use the differential evolution algorithm to optimize the model parameters to achieve closed-loop iteration.

[0047] Through the construction of a multimodal data fusion and decision-making system covering the entire supply chain, the present invention realizes end-to-end intelligent optimization. In the data collection stage, integrate multi-source heterogeneous data such as the Internet of Things and ERP, effectively breaking the information silos in the traditional supply chain. For example, the time-series data obtained through the logistics tracking system can reflect the real-time changes in the location of goods. Combined with the warehouse space data, it can anticipate the sorting pressure in advance.

[0048] The introduction of location encoding and cross-modal attention mechanism in the multimodal fusion model significantly improves the data alignment accuracy. For example, when associating the "urgent additional order" in the voice communication record with the corresponding order image, the accuracy of weight allocation is improved.

[0049] The dynamic strategies generated through reinforcement learning (such as inventory allocation under sudden demand) can improve the order fulfillment rate, and the feedback closed-loop based on blockchain ensures the continuous optimization of the model, avoiding decision failures caused by data drift.

[0050] The overall solution compresses the supply chain response speed from the hour level to the minute level, greatly reducing the comprehensive operating cost and accelerating the response speed in the business process.

[0051] The data preprocessing described in Step 2 includes:

[0052] Use a layout analysis model to locate key areas in the image data to obtain the product name and quantity;

[0053] For voice data, eliminate homophonic ambiguities through a dialect recognition module and commodity library mapping;

[0054] Use entity recognition technology to extract supply chain entities from text data. The supply chain entities include suppliers, SKUs, and delivery times.

[0055] The above technical solution proposes one of the optimization solutions for the preprocessing of multimodal data, significantly improving the reliability of subsequent analysis. For image processing, a layout analysis model (such as PicoDet_layout_1x) is used to accurately locate key fields in logistics documents (such as barcodes and signature dates), avoiding the recognition error rate caused by document wrinkles in traditional OCR (Optical Character Recognition). Voice data is mapped through dialect recognition and a product library to solve the ambiguity problem of the same sentence in different scenarios, such as the ambiguity of "apple" in the fresh food and electronics categories, improving the semantic parsing accuracy. The preprocessing method significantly reduces the interference of noisy data on the fusion model and improves the confidence of the input data for the subsequent decision-making module.

[0056] The position-sensitive optimized multimodal fusion model described in step three specifically includes a cross-modal alignment module, the ERNIE model, and a transition probability matrix.

[0057] The cross-modal alignment module is based on position encoding and is used to eliminate the spatial deviation of different data sources.

[0058] The ERNIE model has a hierarchical processing architecture and is used to achieve efficient semantic parsing.

[0059] The transition probability matrix is used to predict future risk probabilities by combining historical supply chain status data.

[0060] For example, when analyzing a customer's voice complaint of "goods damaged", this mechanism will first associate the corresponding batch of logistics image data (such as photos of damaged outer packaging) rather than irrelevant inventory text records, improving the association accuracy.

[0061] The deep reinforcement learning algorithm described in step four uses the Proximal Policy Optimization framework, which has a reward function, and the reward function is:

[0062] R = a × order fulfillment rate + b × inventory turnover rate - c × logistics delay penalty;

[0063] where a, b, and c are weight coefficients, R is the reward value, and a, b, and c are dynamically adjusted through historical data;

[0064] The turnover efficiency of goods is evaluated according to the R value.

[0065] As one of the implementation scenarios of the above solution, when a = 0.5, b = 0.3, and c = 0.2, in the Double Eleven promotion scenario, resources can be automatically tilted to ensure the supply of popular products, greatly improving the order fulfillment rate. At the same time, the logistics delay rate is controlled at a low level through the logistics delay penalty mechanism.

[0066] The blockchain technology described in step five is specifically:

[0067] Build a consortium blockchain for storing supply chain decision logs and execution results;

[0068] Automatically trigger the data feedback and model iteration process through smart contracts.

[0069] The application of blockchain technology ensures the transparency and traceability of supply chain decisions. When using a consortium blockchain to store data, each node (supplier, logistics provider, retailer) can share decision logs under permission control, solving the trust problem of traditional centralized systems and ensuring traceability in dispute handling.

[0070] For the automatic execution feedback process of smart contracts: when the actual delivery time exceeds the predicted value by 10%, immediately trigger the model parameter adjustment instruction, and shorten the iteration cycle from weekly to hourly.

[0071] It also includes the dynamic path planning step:

[0072] Based on real-time traffic data and delivery requirements, optimize the path selection through a reward and punishment mechanism. If there is congestion or abnormal weather on the predicted path, re-plan the path.

[0073] The dynamic path planning step realizes flexible scheduling through real-time data. When the traffic monitoring API returns that a certain section is congested, the system automatically superimposes a penalty coefficient (such as the original path score × 0.6) and recalculates the optimal path. Combining with the weather API data, avoid low-lying sections in advance during rainstorm warnings.

[0074] The construction method of the transition probability matrix described in step three includes:

[0075] Define the supply chain state set as {S1: normal operation, S2: inventory shortage, S3: logistics delay};

[0076] Calculate the state transition probability based on historical data.

[0077] After defining the state set {S1, S2, S3}, calculate the transition probability according to historical data. When real-time data monitors that the inventory level approaches the safety threshold (for example, when the inventory level is 10%), if the model predicts that the probability of entering the S2 state within the next 72 hours exceeds 30%, immediately trigger an alarm and automatically generate a replenishment strategy.

[0078] The input module is also included in step one. The input module supports uploading data through voice input and picture upload. The customized design for the commercial wholesale scenario greatly improves the practical operation efficiency. Voice input supports real-time dialect translation (such as automatically converting the Cantonese sentence "urgent three boxes of goods" into a standard order), combined with automatic recognition of picture upload (such as scanning the drug batch number), and compresses the order-opening process.

[0079] The technical features of the above embodiments can be combined arbitrarily (as long as there is no contradiction in the combination of these technical features). For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described; these embodiments that are not explicitly written should also be considered to be within the scope described in this specification.

Claims

1. An optimization method for the commercial circulation supply chain based on multi-modal data fusion, characterized in that, Including the following steps, Step 1, Multimodal data collection: Through Internet of Things devices, enterprise ERP systems, logistics tracking systems, and third-party platforms, real-time collect multimodal data across the entire supply chain. The multimodal data includes text data, image data, voice data, time-series data, and spatial data. The text data includes order information and contract terms. The image data includes product pictures and logistics documents. The voice data includes call communication records with customers. The time-series data includes inventory changes and transportation trajectories. The spatial data includes warehouse locations and delivery routes; Step 2, Data preprocessing and feature extraction: Clean, standardize, and denoise the multimodal data. Use a Transformer-based multimodal encoder to extract features from text, image, and voice data to generate high-dimensional semantic vectors. The time-series data extracts dynamic features through an LSTM network, and the spatial data is converted into coordinate vectors through geocoding; Step 3, Multimodal data fusion and optimization: Build a position-sensitive optimized multimodal fusion model. Input the feature vectors extracted in Step 2 into the fusion model. Calculate the correlation weights of text-image, text-voice, time-series-spatial data through a cross-modal attention mechanism to generate a unified feature matrix after fusion. The fusion model combines the transition probability matrix to predict the supply chain state and dynamically adjusts the weight allocation according to real-time data; Step 4, Supply chain intelligent decision-making: Based on the unified feature matrix, use a deep reinforcement learning algorithm to generate supply chain optimization strategies. The supply chain optimization strategies include dynamic inventory allocation, route planning, supplier matching, and risk warning; Step 5, Feedback and iteration: Record the execution results of supply chain decisions through blockchain technology, feedback the actual benefit data to the fusion model, and use the differential evolution algorithm to optimize the model parameters to achieve closed-loop iteration.

2. The optimization method for a business circulation supply chain based on multi-modal data fusion according to claim 1, wherein The data preprocessing described in Step 2 includes: Use a layout analysis model to locate key areas in the image data to obtain product names and quantities; For voice data, eliminate homophone ambiguities through a dialect recognition module and product library mapping; Use entity recognition technology to extract supply chain entities from text data. The supply chain entities include suppliers, SKUs, and delivery times.

3. A method for optimizing a business circulation supply chain based on multi-modal data fusion according to claim 1, characterized in that, The position-sensitive optimized multimodal fusion model described in Step 3 specifically includes a cross-modal alignment module, a Wenxin large model, and a transition probability matrix; The cross-modal alignment module is based on position encoding and is used to eliminate spatial deviations of different data sources; The Wenxin large model has a hierarchical processing architecture and is used to achieve efficient semantic parsing; The transition probability matrix is used to predict future risk probabilities by combining historical supply chain state data.

4. A method for optimizing a business circulation supply chain based on multi-modal data fusion according to claim 1, characterized in that, The deep reinforcement learning algorithm described in Step 4 uses a proximal policy optimization framework, which has a reward function. The reward function is: R = a × order fulfillment rate + b × inventory turnover rate - c × logistics delay penalty; Where a, b, and c are weight coefficients, R is the reward value, and a, b, and c are dynamically adjusted through historical data; Evaluate the turnover efficiency of products according to the R value.

5. A method for optimizing a business circulation supply chain based on multi-modal data fusion according to claim 1, characterized in that, The blockchain technology described in Step 5 specifically is: Build a consortium chain for storing supply chain decision logs and execution results; Automatically trigger the data feedback and model iteration process through smart contracts.

6. A method for optimizing the business circulation supply chain based on multi-modal data fusion according to claim 1, characterized in that, It also includes dynamic path planning steps: Based on real-time traffic data and delivery requirements, optimize the path selection through a reward and punishment mechanism. If there is congestion or abnormal weather on the predicted path, re-plan the path.

7. A method for optimizing the business circulation supply chain based on multi-modal data fusion according to claim 1, characterized in that, The construction method of the transition probability matrix described in step three includes: Define the supply chain state set as {S1: normal operation, S2: inventory shortage, S3: logistics delay}; Calculate the state transition probability according to historical data.

8. A method for optimizing a business circulation supply chain based on multi-modal data fusion according to claim 1, characterized in that, The first step also includes an input module, which supports uploading data by voice input and picture upload.

Images