A logistics supply chain multi-modal anomaly prediction method and system based on a hybrid expert model

Abstract

The application provides a logistics supply chain multi-modal anomaly prediction method and system based on a hybrid expert model, the steps of the method comprising: collecting and recording multi-modal data of a logistics supply chain, the multi-modal data comprising text event data, visual data and time series data; based on a preset dynamic time window, the multi-modal data is divided to obtain a plurality of window data, and a modal feature vector is encoded, a missing mask and a prior uncertainty are constructed based on the window data; the modal feature vector, the missing mask and the prior uncertainty are input into a preset gating network, the gating network outputs the routing probability of a plurality of preset expert sub-models, and an expert sub-model is selected, model reasoning is performed based on the selected expert sub-model, the expert sub-model outputs an anomaly score, an anomaly type and a posterior predictive uncertainty, whether an anomaly occurs is determined based on the anomaly score and the posterior predictive uncertainty, and if an anomaly occurs, an anomaly propagation link is determined.

Description

Technical Field

[0001] This invention relates to the field of logistics supply chain technology, and in particular to a method and system for predicting multimodal anomalies in logistics supply chains based on a hybrid expert model. Background Technology

[0002] In core logistics scenarios, supply chain risks such as order fluctuations, transportation delays, warehouse congestion, temperature control anomalies, and cargo damage or loss are characterized by suddenness, chain-like propagation across links, and the coupling of multiple factors. Currently, enterprises mainly rely on WMS / TMS / ERP systems and IoT sensing devices for risk monitoring. However, the monitoring data collected by these systems suffers from problems such as multi-source heterogeneity, inconsistent sampling frequencies, frequent data gaps, and significant noise interference. Ultimately, this leads to generally delayed risk warnings and high false alarm and missed alarm rates, failing to meet the core requirements of supply chain security management and compliance auditing.

[0003] With the digital upgrade of the supply chain, enterprises can accumulate multimodal data across the entire chain, such as order texts, work logs, RFID time-series data, sensor monitoring data, video surveillance features, vehicle trajectories, and traffic flow data, providing a complete data foundation for building end-to-end intelligent anomaly prediction models.

[0004] However, existing technologies lack multimodal fusion capabilities: they cannot simultaneously and efficiently utilize multimodal information such as text, vision, time series, and graph structure, resulting in extremely poor prediction stability and high false positive and false negative rates when facing real business scenarios with data occlusion, missing data, and weak signals. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a multimodal anomaly prediction method for logistics supply chain based on a hybrid expert model, in order to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present invention provides a method for predicting multimodal anomalies in a logistics supply chain based on a hybrid expert model, the method comprising the following steps: Collect and record multimodal data of the logistics supply chain, including text event data, visual data, and time-series data; Multimodal data is segmented based on a preset dynamic time window to obtain multiple window data. The text event data, visual data, and time series data of the window data are encoded to obtain modal feature vectors. Based on the missing data of the window data, a missing mask is constructed for each modality. Prior uncertainty is constructed based on the missing mask. The modal feature vector, missing mask, and prior uncertainty are input into a preset gating network. The gating network outputs the routing probabilities of multiple preset expert sub-models. An expert sub-model is selected based on the routing probabilities. Model inference is performed based on the selected expert sub-model. The output of the expert sub-model includes anomaly score, anomaly type, and posterior prediction uncertainty. An anomaly is determined based on the anomaly score and posterior prediction uncertainty. A supply chain event graph is constructed based on supply chain events statistically analyzed using a preset dynamic time window. The supply chain event graph includes nodes corresponding to supply chain events and edges constructed based on event relationships. If an anomaly occurs, a path search is performed based on the aforementioned supply chain event graph to determine the anomaly propagation link.

[0007] By adopting the above-mentioned approach, this solution deeply adapts the hybrid expert architecture to the characteristics of the supply chain scenario, and constructs a closed-loop technical system from heterogeneous data processing and dynamic routing reasoning to interpretable root cause localization. It solves the technical problem faced by traditional supply chain anomaly prediction solutions at the underlying architecture level, which is unable to simultaneously and efficiently utilize multimodal information from text, vision, time series, and graph structures. By setting missing masks and prior uncertainties, this solution maintains prediction stability and reduces false alarm and false negative rates when facing real business scenarios with data occlusion, missing data, and weak signals.

[0008] In some embodiments of the present invention, in the step of constructing a missing mask based on the missing window data corresponding to the data of each modality, and constructing a priori uncertainty based on the missing mask; For text event data, determine whether there is a valid event within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. For time-series data, determine whether there are valid sampling points of the sensor within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. For visual data, determine whether there are valid keyframes within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. The missing mask values ​​of the text event data, visual data, and time series data are combined to obtain the missing mask, and the prior uncertainty is determined based on the value of each dimension in the missing mask.

[0009] In some embodiments of the present invention, in the step of determining the prior uncertainty based on the value of each dimension in the missing mask, the prior uncertainty is calculated using the following formula: in, For dynamic time windows The prior uncertainty, This represents the total number of dimensions with missing masks. Dimensions of missing mask The value of .

[0010] In some embodiments of the present invention, in the step of inputting the modal feature vector, missing mask, and prior uncertainty into a preset gating network, and the gating network outputting the routing probabilities of a preset plurality of expert sub-models, the modal feature vector is fed into the preset gating network to obtain the original routing probabilities of the plurality of expert sub-models, the original routing probabilities of the plurality of expert sub-models are combined into an original routing vector, and the original routing vector, missing mask, and prior uncertainty are input again into the softmax layer to obtain the final routing probabilities of the plurality of expert sub-models; the gating network adopts a two-layer MLP structure and is connected to a softmax layer, and the routing probabilities of each expert sub-model are output through the softmax layer.

[0011] In some embodiments of the present invention, the expert sub-models include a text expert model, a visual expert model, a temporal expert model, and a graph structure expert model; the input of the text expert model is the encoding vector of the corresponding text event data in the modal feature vector; the input of the visual expert model is the encoding vector of the corresponding visual data in the modal feature vector; the input of the temporal expert model is the encoding vector of the corresponding temporal data in the modal feature vector; the input of the graph structure expert model is a supply chain event graph, and the result vector output by each of the expert sub-models includes anomaly scores and anomaly type probabilities.

[0012] In some embodiments of the present invention, the text expert model includes a linear projection layer, multiple transformer layers, a layer normalization layer, and an average pooling layer arranged in sequence; the visual expert model includes a linear projection layer, the first 5 layers of MobileNetV3, multiple transformer layers, and an average pooling layer arranged in sequence; the temporal expert model includes a linear projection layer, multiple dilated causal convolutional layers, a temporal attention mechanism layer, and a global max pooling layer arranged in sequence; and the graph structure expert model includes two GAT layers and a global average pooling layer arranged in sequence.

[0013] In some embodiments of the present invention, in the step of determining whether an anomaly has occurred based on the anomaly score and the posterior prediction uncertainty, if the anomaly score is greater than a preset anomaly threshold and the posterior prediction uncertainty is less than a preset uncertainty threshold, then an anomaly is determined to have occurred.

[0014] In some embodiments of the present invention, in the step of performing path search based on the supply chain event graph to determine the anomaly propagation link, the multimodal data is segmented based on the correspondence between the supply chain events and multimodal data to obtain event data corresponding to nodes in the supply chain event graph. The event data is encoded to obtain event vectors, and the node anomaly degree of each node is calculated using the MoE model. Anomaly nodes are selected from the supply chain event graph based on the node anomaly degree, and an anomaly node set is constructed. The anomaly propagation link is determined based on the anomaly nodes in the anomaly node set.

[0015] In some embodiments of the present invention, in the step of determining the abnormal propagation link based on the abnormal nodes in the abnormal node set, an initial path set is constructed based on the edge connection relationship of the abnormal nodes in the supply chain event graph, the path abnormality degree is calculated based on the node abnormality degree of the nodes in each path in the initial path set, and the abnormal propagation link is obtained by filtering from the initial path set based on the path abnormality degree.

[0016] A second aspect of the present invention also provides a multimodal anomaly prediction system for logistics supply chain based on a hybrid expert model. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0017] A third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method for predicting multimodal anomalies in a logistics supply chain based on a hybrid expert model.

[0018] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0019] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0020] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0021] Figure 1This is a schematic diagram of one implementation of the multimodal anomaly prediction method for logistics supply chain based on a hybrid expert model, as presented in this paper. Figure 2 This is a schematic diagram of the processing architecture of this solution; Figure 3 This is a schematic diagram of the processing flow of this solution; Figure 4 This is a schematic diagram of the data structure for the tamper-proof audit records throughout the entire process of this solution. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0023] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0024] like Figure 1 As shown, this invention proposes a multimodal anomaly prediction method for logistics supply chains based on a hybrid expert model. The steps of the method include: Step S100: Collect and record multimodal data of the logistics supply chain, including text event data, visual data, and time-series data; In the specific implementation process, such as Figure 2 As shown, it connects to supply chain WMS / TMS / ERP systems, RFID readers, temperature and humidity / vibration sensors, surveillance cameras, and traffic data platforms to collect multimodal raw data, including order text, work logs, RFID time-series data, sensor time-series data, video / image features, vehicle trajectory data, and traffic flow data, while simultaneously performing data cleaning, desensitization, and standardization preprocessing. Specifically, the text event data includes order text and work logs; the visual data consists of video / image features captured by a camera; the time-series data includes sensor time-series data, vehicle trajectory data, and traffic flow data collected by various sensors, where the vehicle trajectory data is the vehicle's path, and the traffic flow data includes the vehicle's speed and dwell time.

[0025] Step S200: The multimodal data is segmented based on a preset dynamic time window to obtain multiple window data. The text event data, visual data and time series data of the window data are encoded respectively to obtain modal feature vectors. A missing mask is constructed based on the missing data of each modality. A prior uncertainty is constructed based on the missing mask. Specifically, the dynamic time window is a time window triggered based on core events (such as warehousing and distribution) as anchor points, and different core events correspond to time windows of different lengths.

[0026] In the specific implementation process, in the step of encoding the text event data, visual data and time series data of the window data to obtain modal feature vectors, the text event data adopts a 6-layer Transformer encoder, the visual data adopts a MobileNetV3+3-layer ViT network architecture, and the time series data adopts a 3-layer TCN network architecture; the encoded data of each window data are concatenated to obtain the modal feature vector corresponding to each dynamic time window.

[0027] Step S300: Input the modal feature vector, missing mask and prior uncertainty into a preset gating network. The gating network outputs the routing probabilities of multiple preset expert sub-models. Select an expert sub-model based on the routing probabilities. Perform model inference based on the selected expert sub-model. The output of the expert sub-model includes anomaly score, anomaly type and posterior prediction uncertainty. Determine whether an anomaly has occurred based on the anomaly score and posterior prediction uncertainty. Step S400: Construct a supply chain event graph based on the supply chain events statistically analyzed by a preset dynamic time window. The supply chain event graph includes nodes corresponding to supply chain events and edges constructed based on event relationships. In the specific implementation process, in the step of constructing a supply chain event graph based on the supply chain events statistically analyzed by a preset dynamic time window, the nodes include five types of nodes: order nodes, warehousing nodes, transportation nodes, supplier nodes, and abnormal event nodes, and are accompanied by exclusive business attributes; the edges include causal dependency edges based on business associations and temporal propagation edges based on risk propagation associations, and are accompanied by weight attributes.

[0028] In step S500, if an anomaly occurs, a path search is performed based on the supply chain event graph to determine the anomaly propagation link.

[0029] By adopting the above-mentioned approach, this solution deeply adapts the hybrid expert architecture to the characteristics of the supply chain scenario, and constructs a closed-loop technical system from heterogeneous data processing and dynamic routing reasoning to interpretable root cause localization. It solves the technical problem faced by traditional supply chain anomaly prediction solutions at the underlying architecture level, which is unable to simultaneously and efficiently utilize multimodal information from text, vision, time series, and graph structures. By setting missing masks and prior uncertainties, this solution maintains prediction stability and reduces false alarm and false negative rates when facing real business scenarios with data occlusion, missing data, and weak signals.

[0030] In some embodiments of the present invention, in the step of constructing a missing mask based on the missing window data corresponding to the data of each modality, and constructing a priori uncertainty based on the missing mask; For text event data, determine whether there is a valid event within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. In the specific implementation process, in the step of determining whether there is a valid event within the dynamic time window for text event data, if there is order text in the window and the event corresponding to the order text is in the preset set of valid events, then there is a valid event.

[0031] For time-series data, determine whether there are valid sampling points of the sensor within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. In the specific implementation process, for time series data, it is determined whether there are valid sampling points of the sensor within the dynamic time window, and whether the sampling value of the sampling point is within the preset sampling range. If so, there are valid sampling points.

[0032] For visual data, determine whether there are valid keyframes within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. In the specific implementation process, in the step of determining whether there are valid keyframes within the dynamic time window of visual data, it is necessary to identify whether there is a valid subject in the video frame. If there is, then there is a valid keyframe. The valid subject can be a person or a vehicle, etc.

[0033] The missing mask values ​​of the text event data, visual data, and time series data are combined to obtain the missing mask, and the prior uncertainty is determined based on the value of each dimension in the missing mask.

[0034] In the specific implementation process, there is a missing mask for each dynamic time window. By combining the missing masks of all dynamic time windows, a missing mask matrix is ​​obtained.

[0035] In some embodiments of the present invention, in the step of determining the prior uncertainty based on the value of each dimension in the missing mask, the prior uncertainty is calculated using the following formula: in, For dynamic time windows The prior uncertainty, This represents the total number of dimensions with missing masks. Dimensions of missing mask The value of .

[0036] By adopting the above scheme, this scheme unifies heterogeneous data such as discrete events, continuous time series, and visual data into standardized feature sequences through time alignment. At the same time, it generates a dimension-wise missing mask and prior uncertainty, realizing the unified quantitative representation of data quality and context preservation. From the data input level, it adapts to the common characteristics of heterogeneous data, high missing rate, and high noise in the supply chain scenario, effectively avoiding the interference of low-quality data on the subsequent feature extraction and inference process. The data quality quantification results directly participate in the routing decision, enabling the model to dynamically adjust the processing strategy according to the input quality. Meanwhile, the standardized feature design significantly reduces the data adaptation cost for different business scenarios.

[0037] In some embodiments of the present invention, in the step of inputting the modal feature vector, missing mask, and prior uncertainty into a preset gating network, and the gating network outputting the routing probabilities of a preset plurality of expert sub-models, the modal feature vector is fed into the preset gating network to obtain the original routing probabilities of the plurality of expert sub-models, the original routing probabilities of the plurality of expert sub-models are combined into an original routing vector, and the original routing vector, missing mask, and prior uncertainty are input again into the softmax layer to obtain the final routing probabilities of the plurality of expert sub-models; the gating network adopts a two-layer MLP structure and is connected to a softmax layer, and the routing probabilities of each expert sub-model are output through the softmax layer.

[0038] In some embodiments of the present invention, the expert sub-models include a text expert model, a visual expert model, a temporal expert model, and a graph structure expert model; the input of the text expert model is the encoding vector of the corresponding text event data in the modal feature vector; the input of the visual expert model is the encoding vector of the corresponding visual data in the modal feature vector; the input of the temporal expert model is the encoding vector of the corresponding temporal data in the modal feature vector; the input of the graph structure expert model is a supply chain event graph, and the result vector output by each of the expert sub-models includes anomaly scores and anomaly type probabilities.

[0039] In some embodiments of the present invention, the text expert model includes a linear projection layer, multiple transformer layers, a layer normalization layer, and an average pooling layer arranged in sequence; the visual expert model includes a linear projection layer, the first 5 layers of MobileNetV3, multiple transformer layers, and an average pooling layer arranged in sequence; the temporal expert model includes a linear projection layer, multiple dilated causal convolutional layers, a temporal attention mechanism layer, and a global max pooling layer arranged in sequence; and the graph structure expert model includes two GAT layers and a global average pooling layer arranged in sequence.

[0040] By adopting the above approach, this solution freezes the encoder parameters of the general modality expert in a cold start scenario, only fine-tuning a small number of parameters of the gating network and calibration layer. It utilizes business feedback data to quickly adapt to new scenario features, which can significantly reduce the amount of labeled data and training computing power required for new scenario adaptation, effectively shortening the adaptation cycle. At the same time, it avoids the catastrophic forgetting problem caused by full retraining, ensuring that the prediction accuracy of historical scenarios is not affected by the adaptation to new scenarios, and adapting to the iterative needs of rapid expansion of supply chain business.

[0041] like Figure 3 As shown, in some embodiments of the present invention, in the step of determining whether an anomaly has occurred based on the anomaly score and the posterior prediction uncertainty, if the anomaly score is greater than a preset anomaly threshold and the posterior prediction uncertainty is less than a preset uncertainty threshold, then an anomaly is determined to have occurred.

[0042] In some embodiments of the present invention, in the step of performing path search based on the supply chain event graph to determine the anomaly propagation link, the multimodal data is segmented based on the correspondence between the supply chain events and multimodal data to obtain event data corresponding to nodes in the supply chain event graph. The event data is encoded to obtain event vectors, and the node anomaly degree of each node is calculated using the MoE model. Anomaly nodes are selected from the supply chain event graph based on the node anomaly degree, and an anomaly node set is constructed. The anomaly propagation link is determined based on the anomaly nodes in the anomaly node set.

[0043] In some embodiments of the present invention, in the step of determining the abnormal propagation link based on the abnormal nodes in the abnormal node set, an initial path set is constructed based on the edge connection relationship of the abnormal nodes in the supply chain event graph, the path abnormality degree is calculated based on the node abnormality degree of the nodes in each path in the initial path set, and the abnormal propagation link is obtained by filtering from the initial path set based on the path abnormality degree.

[0044] In the specific implementation process, if the node anomaly degree is greater than the node anomaly threshold, it is determined to be an abnormal node. Based on the abnormal node, the supply chain event graph is expanded through the edge connection relationship to obtain multiple initial paths. The calculation weight is determined based on the distance to the abnormal node in the initial path. The node anomaly degree of the nodes in the initial path is weighted based on the calculation weight to determine the path anomaly degree. Based on the path anomaly degree, the initial path with the largest preset number of path anomalies is selected from the initial path set as the anomaly propagation link.

[0045] like Figure 4As shown, in the specific implementation process, after determining the abnormal propagation link, a full-process anti-tampering audit record is generated and stored using a hash chain structure to ensure immutability and traceability; abnormal handling results, manual review and annotation, and business KPI data are collected and written into the training sample library; incremental model training and online calibration are performed regularly, and the gating network and calibration layer parameters are updated first, and the basic expert encoder is frozen to achieve rapid cold start adaptation for new scenarios and avoid catastrophic amnesia.

[0046] Using the above approach, this solution calculates the anomaly scores of all nodes in batches based on global features and filters high-risk targets. Then, for each target node, it traces back the upstream propagation path to determine the final root cause. This solution constructs a complete and interpretable link from global anomaly warning to specific root cause location, avoiding the shortcomings of traditional black-box models that only output results but cannot trace the cause. The path pruning and deduplication mechanism effectively reduces invalid calculations and ensures retrieval efficiency in large-scale event graphs. The output structured propagation link can be directly connected to existing business systems without secondary processing to support anomaly handling and compliance auditing.

[0047] This invention also provides a multimodal anomaly prediction system for logistics supply chain based on a hybrid expert model. The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0048] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned multimodal anomaly prediction method for logistics supply chains based on hybrid expert models. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0049] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0050] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0051] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0052] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting multimodal anomalies in a logistics supply chain based on a hybrid expert model, characterized in that, The steps of the method include: Collect and record multimodal data of the logistics supply chain, including text event data, visual data, and time-series data; Multimodal data is segmented based on a preset dynamic time window to obtain multiple window data. The text event data, visual data, and time series data of the window data are encoded to obtain modal feature vectors. Based on the missing data of the window data, a missing mask is constructed for each modality. Prior uncertainty is constructed based on the missing mask. The modal feature vector, missing mask, and prior uncertainty are input into a preset gating network. The gating network outputs the routing probabilities of multiple preset expert sub-models. An expert sub-model is selected based on the routing probabilities. Model inference is performed based on the selected expert sub-model. The output of the expert sub-model includes anomaly score, anomaly type, and posterior prediction uncertainty. An anomaly is determined based on the anomaly score and posterior prediction uncertainty. In the step of inputting the modal feature vector, missing mask, and prior uncertainty into a preset gating network, and the gating network outputting the routing probabilities of multiple expert sub-models, the modal feature vector is fed into the preset gating network to obtain the original routing probabilities of multiple expert sub-models. The original routing probabilities of multiple expert sub-models are combined into an original routing vector. The original routing vector, missing mask, and prior uncertainty are then input into the softmax layer to obtain the final routing probabilities of multiple expert sub-models. The gating network adopts a two-layer MLP structure and is connected to a softmax layer, through which the routing probabilities of each expert sub-model are output. A supply chain event graph is constructed based on supply chain events statistically analyzed using a preset dynamic time window. The supply chain event graph includes nodes corresponding to supply chain events and edges constructed based on event relationships. If an anomaly occurs, a path search is performed based on the aforementioned supply chain event graph to determine the anomaly propagation link.

2. The method for predicting multimodal anomalies in logistics supply chains based on a hybrid expert model according to claim 1, characterized in that, In the steps of constructing a missing mask for each modality based on the missing window data, and constructing prior uncertainty based on the missing mask; For text event data, determine whether there is a valid event within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. For time-series data, determine whether there are valid sampling points of the sensor within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. For visual data, determine whether there are valid keyframes within the dynamic time window. If yes, set the corresponding dimension of the missing mask to 1; otherwise, set the corresponding dimension of the missing mask to 0. The missing mask values ​​of the text event data, visual data, and time series data are combined to obtain the missing mask, and the prior uncertainty is determined based on the value of each dimension in the missing mask.

3. The method for predicting multimodal anomalies in logistics supply chains based on a hybrid expert model according to claim 2, characterized in that, In the step of determining the prior uncertainty based on the value of each dimension in the missing mask, the prior uncertainty is calculated using the following formula: in, For dynamic time windows The prior uncertainty, This represents the total number of dimensions with missing masks. Dimensions of missing mask The value of .

4. The method for predicting multimodal anomalies in logistics supply chains based on a hybrid expert model according to claim 1, characterized in that, The expert sub-models include a text expert model, a visual expert model, a temporal expert model, and a graph structure expert model. The input of the text expert model is the encoding vector of the corresponding text event data in the modal feature vector. The input of the visual expert model is the encoding vector of the corresponding visual data in the modal feature vector. The input of the temporal expert model is the encoding vector of the corresponding temporal data in the modal feature vector. The input of the graph structure expert model is a supply chain event graph. The output vector of each expert sub-model includes anomaly score and anomaly type probability.

5. The method for predicting multimodal anomalies in logistics supply chains based on a hybrid expert model according to claim 4, characterized in that, The text expert model includes a linear projection layer, multiple transformer layers, a layer normalization layer, and an average pooling layer arranged in sequence; the visual expert model includes a linear projection layer, the first 5 layers of MobileNetV3, multiple transformer layers, and an average pooling layer arranged in sequence; the temporal expert model includes a linear projection layer, multiple dilated causal convolutional layers, a temporal attention mechanism layer, and a global max pooling layer arranged in sequence; the graph structure expert model includes two GAT layers arranged in sequence and a global average pooling layer.

6. The method for predicting multimodal anomalies in logistics supply chains based on a hybrid expert model according to claim 1, characterized in that, In the step of determining whether an anomaly has occurred based on the anomaly score and the posterior prediction uncertainty, if the anomaly score is greater than a preset anomaly threshold and the posterior prediction uncertainty is less than a preset uncertainty threshold, then an anomaly is determined to have occurred.

7. The method for predicting multimodal anomalies in logistics supply chains based on a hybrid expert model according to any one of claims 1 to 6, characterized in that, In the step of path search based on the supply chain event graph to determine the abnormal propagation link, the multimodal data is segmented based on the correspondence between the supply chain events and the multimodal data to obtain the event data corresponding to the nodes in the supply chain event graph. The event data is encoded to obtain the event vector, and the node anomaly degree of each node is calculated using the MoE model. Based on the node anomaly degree, abnormal nodes are selected from the supply chain event graph, and an abnormal node set is constructed; The abnormal propagation path is determined based on the abnormal nodes in the set of abnormal nodes.

8. The method for predicting multimodal anomalies in logistics supply chains based on a hybrid expert model according to claim 7, characterized in that, In the step of determining the abnormal propagation link based on the abnormal nodes in the abnormal node set, an initial path set is constructed based on the edge connection relationship of the abnormal nodes in the supply chain event graph, the path abnormality degree is calculated based on the node abnormality degree of the nodes in each path in the initial path set, and the abnormal propagation link is obtained by filtering from the initial path set based on the path abnormality degree.

9. A multimodal anomaly prediction system for logistics supply chain based on a hybrid expert model, characterized in that, The system includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps of the method as described in any one of claims 1 to 8.

Images