Light industry supply chain multi-modal data fusion analysis method based on deep learning

Abstract

The invention discloses a light industry supply chain multi-modal data fusion analysis method based on deep learning, and the method comprises the following steps: carrying out the cleaning and standardization processing of text, image, audio and video data collected in a supply chain environment, and constructing a standardized multi-modal data set; then, a special feature extraction network is adopted to generate each modal feature vector, and a feature incidence matrix is constructed through cross-modal correlation analysis; feature weights are dynamically adjusted in combination with a domain knowledge rule base, multi-modal feature interaction is achieved through a cross-modal attention fusion network, and unified fusion features are generated through a self-attention mechanism; and finally, constructing a supply chain decision model, and mapping the fusion feature into a supply chain state evaluation result and an optimization parameter. According to the method, knowledge rule constraint and a deep attention mechanism are fused, supply chain situation awareness precision and decision reliability can be effectively improved, and technical support is provided for intelligent management of the light industry supply chain.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a light industry supply chain multimodal data fusion analysis method based on deep learning. Background Art

[0002] With the digital transformation of manufacturing, light industry is facing profound changes in production models and management methods. During this transformation, supply chain management, as a core component of business operations, has a direct impact on a company's market competitiveness through its intelligent level. Light industry supply chains are characterized by a wide variety of products, short production cycles, and rapidly changing market demand. This generates a large amount of multimodal data within the supply chain, including text order information, product images, equipment operating sounds, production line video surveillance, and structured data collected by various sensors. This heterogeneous data contains rich business information and value. How to effectively process, analyze, and mine this multimodal data has become a key challenge in intelligent supply chain management.

[0003] In recent years, deep learning technology has achieved breakthroughs in areas such as image recognition, natural language processing, and speech recognition, providing new technical means for multimodal data analysis. Traditional supply chain data analysis relies primarily on statistical methods and simple machine learning algorithms, which have difficulty processing unstructured data, and the analysis results are often limited to a single data source. With the development of deep learning technology, models such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and Transformers have demonstrated powerful data processing capabilities in their respective fields, laying the technical foundation for multimodal data fusion analysis. In the light industrial supply chain, some advanced companies have begun to experiment with applying deep learning technology to process image and text data within the supply chain. For example, deep learning models are used to automatically identify product features by analyzing product images, or natural language processing technology is used to extract demand characteristics from customer orders to support supply chain decision-making.

[0004] Although deep learning technology has made significant progress in processing single-modal data, existing multimodal data analysis methods for light industrial supply chains still have significant shortcomings. These methods generally lack effective inter-modal fusion mechanisms and are unable to fully explore the inherent connections between different types of data. In practical applications, most models fail to optimize for the specific scenarios of light industry, resulting in a disconnect between analysis results and business needs. Furthermore, the semantic differences between data of different modalities pose a significant challenge in integrating information, making it difficult for existing methods to construct a unified data representation framework. More importantly, current data analysis methods often overlook the value of domain expert knowledge and lack an effective way to organically integrate professional knowledge with data-driven methods, resulting in poor performance when handling complex business logic.

[0005] The above-mentioned technical bottlenecks severely limit the application potential of multimodal data in light industry supply chain optimization, and a more advanced and systematic solution is urgently needed. Summary of the Invention

[0006] The purpose of this invention is to provide a light industry supply chain multimodal data fusion analysis method based on deep learning, which can effectively integrate heterogeneous data and extract deep correlation features to provide accurate support for supply chain decision-making.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a light industry supply chain multimodal data fusion analysis method based on deep learning, comprising the following steps: S1. Collect text data, image data, audio data, and video data in a light industry supply chain environment to form a multimodal raw data set, and perform data cleaning, noise filtering, and format standardization on the multimodal raw data set to obtain a normalized multimodal data set; S2. For the text data, image data, audio data, and video data in the normalized multimodal dataset, respectively, a text feature extraction network, an image feature extraction network, an audio feature extraction network, and a video feature extraction network are used to perform feature extraction to generate corresponding text feature vectors, image feature vectors, audio feature vectors, and video feature vectors; S3. Using a cross-modal correlation analysis algorithm, calculate the correlation coefficients among the text feature vector, the image feature vector, the audio feature vector, and the video feature vector, and construct a cross-modal feature correlation matrix; S4. Based on the pre-built light industry supply chain domain knowledge rule base, domain knowledge constraints are applied to the cross-modal feature correlation matrix, and the weight coefficients of each modal feature vector are dynamically adjusted to obtain a set of multimodal feature vectors after weight adjustment. S5. Construct a cross-modal attention fusion network, take the weight-adjusted multimodal feature vector set as input, calculate the attention weights between the feature vectors of each modality through the self-attention mechanism, realize the interactive fusion of cross-modal feature information, and output a unified fusion feature vector; S6. Build a supply chain decision analysis model, use the unified fusion feature vector as input feature, generate supply chain status evaluation results and supply chain optimization configuration parameters through neural network mapping, and finally generate a supply chain optimization plan.

[0008] Preferably, the specific steps of step S2 are: S2.1. Performing a spatiotemporal alignment operation on the text data, image data, audio data, and video data in the normalized multimodal dataset. By extracting the timestamp information and spatial coordinate information of each modal data, a spatiotemporal mapping relationship between the multimodal data is established. Different modal data describing the same supply chain event are correlated and matched to form a spatiotemporally aligned multimodal dataset. S2.2. For the text data in the spatiotemporally aligned multimodal dataset, a BERT pre-trained language model based on the Transformer architecture is used to extract global semantic features. The text sequence information is processed through a bidirectional long short-term memory network to capture the contextual semantic dependencies in the supply chain text data and generate a fixed-dimensional text feature vector. S2.3. For the image data in the spatiotemporally aligned multimodal dataset, a ResNet-50 deep convolutional neural network architecture is used to extract multi-scale visual features of the images through a residual connection mechanism. The edge, texture, shape, and spatial layout feature information of the supply chain scene images are extracted layer by layer to generate corresponding image feature vectors. S2.4. Extracting spectral features of the audio data in the spatiotemporally aligned multimodal dataset using a one-dimensional convolutional neural network. Inputting the spectral features into a long short-term memory network to process audio temporal dependencies, identify sound pattern features in the supply chain environment, and generate acoustic feature vectors representing the audio information. S2.5. For the video data in the spatiotemporally aligned multimodal dataset, use a three-dimensional convolutional neural network to simultaneously extract the spatial and temporal features of the video, combine it with a multi-head self-attention mechanism to identify key action sequences in the supply chain operation process, and generate a video feature vector containing spatiotemporal information; S2.6. Perform L2 norm normalization on each of the text feature vector, image feature vector, acoustic feature vector, and video feature vector to normalize the modulus of each modal feature vector to unit length, thereby eliminating differences in numerical dimensions between feature vectors of different modalities. S2.7. Combining the normalized modal feature vectors to form a multimodal feature vector set, which serves as input data for the cross-modal feature correlation analysis in step S3.

[0009] Preferably, the specific steps of step S3 are: S3.1. Receive the multimodal feature vector set, construct a cross-modal feature pair combination, and combine the feature vectors of any two different modalities to form a binary feature pair, forming a total of N(N-1) / 2 feature pairs, where N is the total number of modalities; S3.2. Perform a dimensional alignment operation on the binary feature pairs, map the feature vectors of different dimensions to a unified dimensional space through a fully connected neural network layer, calculate the linear correlation coefficient of each feature pair using a canonical correlation analysis algorithm, and construct a linear correlation coefficient matrix; S3.3. Construct a cross-modal semantic encoding network, perform semantic space mapping on the binary feature pairs, train a semantic encoder by maximizing the similarity of different modal features of the same supply chain event and minimizing the similarity of different modal features of different supply chain events; calculate the semantic similarity between feature vectors based on the trained semantic encoder, and construct a semantic similarity matrix; S3.4. Based on the linear correlation coefficient matrix and the semantic similarity matrix, calculate the high-order nonlinear correlation relationship between cross-modal features through a deep neural network to generate a high-order correlation matrix; S3.5. Assign weight coefficients α, β, and γ to the linear correlation coefficient matrix, semantic similarity matrix, and high-order relevance matrix, where α + β + γ = 1; adaptively learn the weight coefficients through a weight optimization network, and calculate a comprehensive relevance index using a weighted linear combination; S3.6. Construct an N×N dimensional cross-modal feature correlation matrix based on the comprehensive correlation index, where the matrix elements represent the correlation strength between different modalities and the diagonal elements represent the autocorrelation strength within the modalities; S3.7. Perform standardization on the cross-modal feature correlation matrix, use the minimum-maximum normalization method to map the correlation value to the [0,1] interval, and perform noise correlation suppression to generate a standardized cross-modal feature correlation matrix as input data for step S4.

[0010] Preferably, the specific steps of step S4 are: S4.1. Receive the standardized cross-modal feature correlation matrix output from step S3 and call a pre-built light industry supply chain domain knowledge rule base, which includes supply chain node association rules, logistics process constraint rules, resource allocation rules, and business logic rules. S4.2. Design a knowledge rule matching algorithm to match the association patterns in the standardized cross-modal feature correlation matrix with various rules in the pre-built light industry supply chain domain knowledge rule library, identifying valid associations that conform to the supply chain business logic and marking abnormal associations that do not conform to the business logic. S4.3. Based on the knowledge rule matching results, a rule constraint function is constructed to set suppression weights for relationships marked as abnormal, and enhancement weights for relationships that conform to business logic, thereby generating a rule constraint weight matrix. S4.4. Design an adaptive weight adjustment mechanism to dynamically calculate the importance weight coefficients of the text feature vector, image feature vector, acoustic feature vector, and video feature vector based on the rule constraint weight matrix and the current supply chain status information, and generate a modal feature weight adjustment matrix; S4.5. Performing an element-by-element weight fusion operation on the modal feature weight adjustment matrix and each modal feature vector after normalization output from step S2 to achieve dynamic adjustment of feature weights based on the constraints of the pre-built light industry supply chain domain knowledge rule base; S4.6. Perform feature consistency verification on the text feature vector, image feature vector, acoustic feature vector, and video feature vector after weight adjustment to ensure that the adjusted feature vector maintains the integrity of the original semantic information, and obtain the multimodal feature vector set after weight adjustment as the input data of step S5.

[0011] Preferably, the specific steps of step S5 are: S5.1. Receive the weight-adjusted multimodal feature vector set outputted in step S4, including the weight-adjusted text feature vector, image feature vector, acoustic feature vector, and video feature vector; S5.2. Construct a cross-modal attention fusion network, which includes a query vector generation module, a key-value vector generation module, and an attention weight calculation module; S5.3. Input the weight-adjusted multimodal feature vector set into the query vector generation module and key-value vector generation module of the cross-modal attention fusion network to generate corresponding query vector matrices and key-value vector matrices, respectively; S5.4. Calculating, by an attention weight calculation module, a cross-modal attention weight matrix based on the query vector matrix and the key-value vector matrix, where the cross-modal attention weight matrix represents the importance relationship between feature vectors of different modalities; S5.5. Based on the cross-modal attention weight matrix, perform a weighted fusion operation on the weight-adjusted multimodal feature vector set to achieve interactive fusion of cross-modal feature information; S5.6. Perform dimensionality transformation and nonlinear mapping on the fused feature representation through a fully connected neural network layer to generate a unified fused feature vector of fixed dimension; S5.7. Standardize the unified fusion feature vector to ensure the numerical stability of the feature vector, and output the standardized unified fusion feature vector as input data for step S6.

[0012] Preferably, the specific steps of step S6 are: S6.1. Receive the standardized unified fusion feature vector output from step S5 and construct a supply chain decision analysis model. The supply chain decision analysis model includes a supply chain status assessment module and a supply chain optimization decision module. S6.2. Input the standardized unified fusion feature vector into the supply chain status assessment module, analyze the current supply chain operation status through a multi-layer neural network, and identify the operating efficiency, resource utilization, and risk level of each link in the supply chain; S6.3. Generate a supply chain status assessment report based on the analysis results of the supply chain status assessment module. The supply chain status assessment report includes overall supply chain health indicators, performance indicators of each node, and potential risk warning information; S6.4. Input the standardized unified fusion feature vector and the supply chain status assessment report into the supply chain optimization decision module, and calculate the optimal supply chain configuration parameters using a deep neural network; S6.5. Generate a supply chain optimization plan based on the optimal supply chain configuration parameters. The supply chain optimization plan includes resource allocation adjustment suggestions, process optimization strategies, and risk control measures. S6.6. Format and output the supply chain status assessment report and the supply chain optimization plan to form a complete supply chain analysis and decision-making result, which is provided to the light industry supply chain management system for decision-making execution.

[0013] Preferably, the multimodal data collection in step S1 specifically includes: Collect text data from the light industrial supply chain environment through the Internet of Things sensor network, including production order information, logistics document information, quality inspection reports, and supplier communication records; Collect image data in a light industrial supply chain environment using industrial camera equipment, including production site images, product quality images, storage status images, and logistics transportation images; Collecting audio data from the light industry supply chain environment using sound collection equipment, including the operating sounds of production equipment, logistics vehicles, and operating environment noise information; Video data in a light industry supply chain environment is collected through video surveillance equipment, and the video data includes production process videos, logistics operation videos, and security monitoring videos.

[0014] Compared with the existing technology, the advantages of the present invention are: this method first establishes a standardized multimodal data foundation by comprehensively collecting text, image, audio and video data in the supply chain environment and preprocessing them; then uses a deep learning model optimized for different modal characteristics to extract the exclusive features of each type of data; then establishes a correlation matrix between different modalities through a cross-modal correlation analysis algorithm to capture the intrinsic connection between multidimensional data; then, the data-driven correlation matrix is ​​integrated with the rule base condensed from the experience of domain experts, and the feature weights are dynamically adjusted to bridge the gap between pure data methods and expert knowledge; on this basis, a cross-modal attention fusion network is constructed, which intelligently identifies key information in different modalities through an adaptive attention mechanism and promotes information interaction between modalities to generate a unified feature vector containing comprehensive supply chain status information; finally, the fused feature vector is input into a specially designed decision model to achieve precise optimization of key links such as inventory management, logistics scheduling, demand forecasting and production planning.

[0015] The advantages of this method are: it can process structured and unstructured data simultaneously, organically integrate complementary information of different modalities, combine the dual advantages of data-driven and domain knowledge, adapt to the changing business environment through dynamic weight adjustment, and ultimately provide a comprehensive, accurate and practical optimization solution for light industrial supply chain management, greatly improving the agility, resilience and efficiency of the supply chain. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0017] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] Example 1: As shown in the figure, a light industry supply chain multimodal data fusion analysis method based on deep learning includes the following steps: S1. Collect text data, image data, audio data, and video data from the light industry supply chain environment to form a multimodal raw data set. Perform data cleaning, noise filtering, and format standardization on the multimodal raw data set to obtain a normalized multimodal data set. S2. For the text data, image data, audio data, and video data in the normalized multimodal dataset, feature extraction is performed using a text feature extraction network, an image feature extraction network, an audio feature extraction network, and a video feature extraction network, respectively, to generate corresponding text feature vectors, image feature vectors, audio feature vectors, and video feature vectors; S3. Using a cross-modal correlation analysis algorithm, calculate the correlation coefficients between text feature vectors, image feature vectors, audio feature vectors, and video feature vectors, and construct a cross-modal feature correlation matrix; S4. Based on the pre-built light industry supply chain domain knowledge rule base, domain knowledge constraints are applied to the cross-modal feature correlation matrix, and the weight coefficients of each modal feature vector are dynamically adjusted to obtain a set of multi-modal feature vectors after weight adjustment. S5. Construct a cross-modal attention fusion network, take the weighted multimodal feature vector set as input, calculate the attention weights between the feature vectors of each modality through the self-attention mechanism, realize the interactive fusion of cross-modal feature information, and output a unified fusion feature vector; S6. Build a supply chain decision analysis model, use the unified fusion feature vector as the input feature, generate the supply chain status evaluation results and supply chain optimization configuration parameters through neural network mapping, and finally generate a supply chain optimization plan.

[0020] This deep learning-based multimodal data fusion analysis method for light industrial supply chains establishes a complete technical chain from data acquisition to decision output, offering significant systemic and practical advantages. The entire method utilizes an end-to-end deep learning framework that automates the complex task of multimodal data fusion, avoiding the drawbacks of traditional methods that require extensive manual feature engineering and rule design. By uniformly processing and fusing data from four different modalities—text, images, audio, and video—this method can obtain more comprehensive and accurate supply chain status information than single-modality analysis.

[0021] During the data preprocessing phase (S1-S2), this method ensures that multimodal data from different sources and formats meet consistent quality standards through standardized data cleaning and standardization processes. Specifically optimized deep learning networks are used for feature extraction based on the characteristics of different modal data. For example, a Transformer architecture is used to capture semantic information for text data, a convolutional neural network is used to extract visual features for image data, one-dimensional convolution combined with a recurrent network is used to process temporal information for audio data, and three-dimensional convolution is used to process spatiotemporal features for video data. This modality-specific feature extraction strategy maximizes the preservation of the original information value of each modal data, laying a solid foundation for subsequent fusion analysis.

[0022] Cross-modal correlation analysis in step S3 calculates the correlation coefficients between different modal feature vectors and constructs a cross-modal feature correlation matrix. This can quantitatively reveal the inherent relationships between different modal data. For example, when a production equipment anomaly occurs, it may manifest simultaneously as a change in the equipment's operating sound (audio modality), an abnormality in the production site image (image modality), and the generation of an anomaly report (text modality). Correlation analysis can identify these cross-modal association patterns, providing a more reliable basis for anomaly detection and early warning.

[0023] Another key advantage of this approach is the dynamic weight adjustment mechanism in step S4, based on a pre-built domain knowledge rule base. This mechanism organically combines human expert experience with a data-driven approach. Traditional purely data-driven approaches are susceptible to data bias, but this approach, by incorporating specialized knowledge rules from the supply chain domain, can constrain and correct unreasonable feature associations. For example, if data analysis reveals a discrepancy between a supplier's textual evaluation and its actual delivery quality image, domain knowledge rules can be used to rationalize and adjust the weight of this association based on historical experience, improving the reliability and business applicability of the analysis results.

[0024] The cross-modal attention fusion network employed in step S5 intelligently identifies the importance of information from different modalities, enabling precise information fusion. This attention mechanism enables the model to adaptively focus on the modal information most important for the current decision, avoiding the information overwhelm that can result from simple average fusion. For example, when analyzing urgent order processing, the system might focus more on logistics videos and order text information, while analyzing quality issues might focus more on product images and test report text. This dynamic attention allocation mechanism improves the relevance and accuracy of fusion results.

[0025] The resulting supply chain decision analysis model (S6) transforms complex multimodal fusion features into concrete business decision support, including supply chain status assessment and optimized configuration parameters, directly serving the actual operational needs of enterprises. This method not only provides an assessment of the current supply chain health but also provides specific optimization recommendations, such as resource allocation adjustments and process improvement strategies. It has strong practical value and commercial application prospects. Compared with traditional qualitative analysis methods, the quantitative analysis results provided by this method are more objective and accurate, providing business managers with a scientific basis for decision-making.

[0026] Example 2: Different from Example 1, the specific steps of step S2 are as follows: S2.1. Perform spatiotemporal alignment on the text, image, audio, and video data in the normalized multimodal dataset. By extracting the timestamp and spatial coordinate information of each modal data, establish a spatiotemporal mapping relationship between the multimodal data. Associate and match different modal data describing the same supply chain event to form a spatiotemporally aligned multimodal dataset. S2.2. For text data in a spatiotemporally aligned multimodal dataset, we use the BERT pre-trained language model based on the Transformer architecture to extract global semantic features. We then process text sequence information through a bidirectional long short-term memory network, capturing contextual semantic dependencies in the supply chain text data and generating fixed-dimensional text feature vectors. S2.3. For image data in the spatiotemporally aligned multimodal dataset, we use the ResNet-50 deep convolutional neural network structure to extract multi-scale visual features of the images through a residual connection mechanism. We extract edge, texture, shape, and spatial layout feature information from supply chain scene images layer by layer, and generate corresponding image feature vectors. S2.4. For audio data in a spatiotemporally aligned multimodal dataset, a one-dimensional convolutional neural network is used to extract the spectral features of the audio signal. These spectral features are then fed into a long short-term memory network to process the temporal dependencies of the audio, identify the sound pattern characteristics in the supply chain environment, and generate acoustic feature vectors representing the audio information. S2.5. For video data in a spatiotemporally aligned multimodal dataset, a 3D convolutional neural network is used to simultaneously extract both spatial and temporal features of the video. This is combined with a multi-head self-attention mechanism to identify key action sequences in the supply chain operation process and generate video feature vectors containing spatiotemporal information. S2.6. Perform L2 norm normalization on the text feature vector, image feature vector, acoustic feature vector, and video feature vector, respectively, normalizing the modulus of each modal feature vector to unit length to eliminate the numerical dimension differences between feature vectors of different modalities. S2.7. Combining the normalized feature vectors of each modality to form a multimodal feature vector set, which serves as input data for the cross-modal feature correlation analysis in step S3.

[0027] In the above steps, the spatiotemporal alignment operation in step S2.1 establishes a spatiotemporal mapping relationship by extracting the timestamps and spatial coordinate information of each modal data. This ensures that different modal data can accurately describe the same supply chain event and avoids analytical bias caused by data mismatch. For example, when an equipment failure occurs in a production workshop, this method can accurately match abnormal sound recordings (audio), equipment status images (images), fault reports (text), and surveillance footage (video) at the same time and location, providing a reliable data foundation for subsequent fusion analysis. This spatiotemporal alignment mechanism is particularly suitable for business scenarios such as supply chains with clear spatiotemporal characteristics, significantly improving the accuracy and reliability of multimodal data fusion.

[0028] Steps S2.2-S2.5 use a deep learning network architecture customized for the characteristics of different modal data. For text data, it combines the global semantic understanding capabilities of the BERT pre-trained model and the sequence modeling advantages of the bidirectional LSTM, which can deeply understand the complex semantic relationships and contextual dependencies in supply chain documents; for image data, the residual connection mechanism of ResNet-50 can effectively solve the gradient vanishing problem of deep networks, and extract multi-scale visual features from low-level edge textures to high-level semantic layouts layer by layer; for audio data, the one-dimensional convolutional network is good at capturing frequency domain characteristics, while the LSTM network is good at processing the temporal dependencies of audio. The combination of the two can fully identify sound patterns in the supply chain environment; for video data, the three-dimensional convolutional network can simultaneously process information in spatial and temporal dimensions, and combined with the multi-head self-attention mechanism, it can intelligently identify key action sequences, which is of great significance for analyzing dynamic scenarios such as production processes and logistics operations.

[0029] The L2 norm normalization process in step S2.6 solves a key technical problem in multimodal fusion - the difference in numerical dimensions of feature vectors of different modalities. Due to the significant differences in the physical properties and numerical ranges of different modal data, direct fusion may cause the features of some modalities to be overwhelmed by other modalities. For example, the numerical range of image feature vectors may be between 0-255, while the numerical range of text embedding vectors may be between -1 and 1. If normalization is not performed, image features may dominate the fusion process, affecting the effective use of other modal information. L2 norm normalization unifies the modulus length of all feature vectors to unit length, ensuring the fairness and balance of each modal feature in the fusion process, and providing a standardized input basis for subsequent correlation analysis and weight calculation.

[0030] This step is designed to fully consider the business characteristics and data features of the light industry supply chain. Supply chain management involves multiple links, multiple data sources, and a variety of business scenarios, requiring complex and diverse data types to be processed. This method effectively captures key information from each link of the supply chain through a refined feature extraction strategy: text features capture business communications and document information, image features reflect on-site conditions and product quality, audio features monitor equipment operation and environmental conditions, and video features record operational processes and dynamic changes. This comprehensive information acquisition capability enables the method to build a more comprehensive and accurate supply chain status perception system, providing the data foundation for subsequent intelligent decision-making.

[0031] In this embodiment, the specific steps of step S3 are: S3.1. Receive a set of multimodal feature vectors and construct a cross-modal feature pair combination. Combine feature vectors of any two different modalities to form a binary feature pair, forming a total of N(N-1) / 2 feature pairs, where N is the total number of modalities. S3.2. Perform dimensional alignment on binary feature pairs. Map feature vectors of different dimensions to a unified dimensional space through a fully connected neural network layer. Use a canonical correlation analysis algorithm to calculate the linear correlation coefficient of each feature pair and construct a linear correlation coefficient matrix. S3.3. Construct a cross-modal semantic encoding network to perform semantic space mapping on binary feature pairs. Train the semantic encoder by maximizing the similarity of different modal features of the same supply chain event and minimizing the similarity of different modal features of different supply chain events. Calculate the semantic similarity between feature vectors based on the trained semantic encoder and construct a semantic similarity matrix. S3.4. Based on the linear correlation coefficient matrix and the semantic similarity matrix, a deep neural network is used to calculate the high-order nonlinear correlation between cross-modal features and generate a high-order correlation matrix. S3.5. Assign weight coefficients α, β, and γ to the linear correlation coefficient matrix, the semantic similarity matrix, and the high-order relevance matrix, where α + β + γ = 1; adaptively learn the weight coefficients through a weight optimization network, and calculate the comprehensive relevance index using a weighted linear combination; S3.6. Construct an N×N-dimensional cross-modal feature correlation matrix based on the comprehensive correlation index, where the matrix elements represent the correlation strength between different modalities, and the diagonal elements represent the autocorrelation strength within the modalities; S3.7. Perform normalization on the cross-modal feature correlation matrix, use the minimum-maximum normalization method to map the correlation value to the [0,1] interval, and perform noise correlation suppression to generate a normalized cross-modal feature correlation matrix as the input data for step S4.

[0032] Step S3.1 uses combinatorial mathematics to construct cross-modal feature pairs, ensuring that all pairwise relationships between modalities are fully covered through N(N-1) / 2 combinations. Taking four modalities as an example, this method constructs six feature pairs: text-image, text-audio, text-video, image-audio, image-video, and audio-video, comprehensively covering all possible cross-modal associations. This exhaustive feature pair construction strategy ensures that no potential cross-modal association patterns are missed, providing a complete data foundation for subsequent correlation analysis. For example, in supply chain anomaly detection, complex cross-modal relationships such as abnormal associations between text reports and equipment sounds, and quality associations between product images and transportation videos can be effectively captured and analyzed.

[0033] Steps S3.2-S3.4 construct a multi-level correlation calculation system from linear to nonlinear, from statistical to semantic, significantly improving the accuracy and comprehensiveness of cross-modal association analysis. First, by calculating linear correlation through dimensional alignment and canonical correlation analysis, the basic statistical correlation between modalities can be captured. Second, through the contrastive learning mechanism of the semantic encoding network, different modal features are mapped to a unified semantic space, which can identify deeper semantic correlations. Finally, by calculating high-order nonlinear correlations through deep neural networks, complex implicit correlation patterns can be discovered. This multi-level analysis strategy can fully characterize the complexity of cross-modal associations. For example, there may be linear rating associations, semantic quality description associations, and more complex nonlinear business logic associations between supplier evaluation text and product quality images. This method can simultaneously capture these different levels of associations.

[0034] The adaptive weight optimization mechanism used in step S3.5 is a key innovation of this method. The weight optimization network automatically learns the optimal combination of the three weight coefficients α, β, and γ, avoiding the subjectivity and inaccuracy that may come with manually setting weights. This mechanism can dynamically adjust the importance weights of linear correlation, semantic similarity, and higher-order associations based on different supply chain scenarios and data characteristics. For example, in a quality control scenario, semantic similarity may be more important, and the system will automatically increase the weight of β; in an equipment monitoring scenario, linear correlation may be more critical, and the system will automatically increase the weight of α. This adaptive adjustment mechanism enables the method to flexibly adapt to different application scenarios, improving its versatility and accuracy.

[0035] The N×N-dimensional cross-modal feature correlation matrix constructed in step S3.6 effectively integrates multi-level correlation information, providing a rich information foundation for subsequent domain knowledge fusion and decision analysis. This matrix not only contains cross-modal correlation strength information but also records the autocorrelation strength within each modality through diagonal elements, forming a complete modal correlation relationship map. This matrix representation facilitates subsequent algorithm processing and analysis and also provides an intuitive data structure for visual analysis and manual interpretation. For example, by analyzing the correlation matrix, managers can intuitively understand which modal information is strongly correlated and which modal information is relatively independent, thereby providing data support for the formulation of supply chain management strategies.

[0036] The standardization and noise suppression mechanism in step S3.7 significantly enhances the robustness and reliability of the method. Minimum-maximum normalization maps correlation values ​​to the [0,1] interval, ensuring numerical consistency and comparability. The noise correlation suppression mechanism filters out spurious correlations that may be caused by data noise or accidental factors, enhancing the credibility of the analysis results. This processing strategy is particularly well-suited for complex business environments like the supply chain, effectively addressing practical issues such as inconsistent data quality and noise interference. For example, in a real supply chain environment, there may be abnormal correlations caused by equipment misreporting, data transmission errors, and other issues. This mechanism can effectively identify and suppress these noisy correlations, ensuring the accuracy of the analysis results.

[0037] The design of step S3 fully considers the business characteristics and application requirements of the light industry supply chain. Supply chain management involves multiple links, multiple data sources, and complex business logic, requiring complex cross-modal relationships. This method, through a systematic correlation analysis framework, can effectively identify and quantify these complex relationships, providing a scientific basis for supply chain optimization decisions. For example, this method can discover the correlation between supplier communication frequency (text) and delivery quality (images), the early warning correlation between production equipment sound (audio) and product defect rate (images), and the service quality correlation between logistics video (video) and customer satisfaction (text). These findings have important practical value for supply chain management.

[0038] In this embodiment, the specific steps of step S4 are: S4.1. Receive the standardized cross-modal feature correlation matrix output from step S3 and call a pre-built light industry supply chain domain knowledge rule base, which includes supply chain node association rules, logistics process constraint rules, resource allocation rules, and business logic rules. S4.2. Design a knowledge rule matching algorithm to match the association patterns in the standardized cross-modal feature correlation matrix with various rules in the pre-built light industry supply chain domain knowledge rule library, identifying valid associations that conform to the supply chain business logic and marking abnormal associations that do not conform to the business logic. S4.3. Based on the knowledge rule matching results, a rule constraint function is constructed to set suppression weights for relationships marked as abnormal, and enhancement weights for relationships that conform to business logic, thereby generating a rule constraint weight matrix. S4.4. Design an adaptive weight adjustment mechanism to dynamically calculate the importance weight coefficients of text feature vectors, image feature vectors, acoustic feature vectors, and video feature vectors based on the rule constraint weight matrix and the current supply chain status information, and generate a modal feature weight adjustment matrix; S4.5. Performing an element-by-element weight fusion operation on the modal feature weight adjustment matrix and the normalized modal feature vectors output from step S2 to achieve dynamic adjustment of feature weights based on the constraints of a pre-built light industry supply chain domain knowledge rule base; S4.6. Perform feature consistency verification on the text feature vector, image feature vector, acoustic feature vector, and video feature vector after weight adjustment to ensure that the adjusted feature vector maintains the integrity of the original semantic information, and obtain the multimodal feature vector set after weight adjustment as the input data of step S5.

[0039] The design of step S4 fully embodies engineering thinking for practical supply chain management applications. This step not only addresses the technical challenges of multimodal data fusion but, more importantly, aligns the technical solution with business requirements. By incorporating a domain knowledge rule base and intelligent constraint mechanisms, this method produces analytical results that are both technologically advanced and business-sounding, significantly enhancing its application value and acceptance in practical supply chain management.

[0040] Specifically, the pre-built light industry supply chain domain knowledge rule base in step S4.1 reflects the important innovation of this method in that it organically combines expert experience with data-driven methods. This rule base covers four core dimensions: supply chain node association rules, logistics process constraint rules, resource allocation rules, and business logic rules, comprehensively covering the key business scenarios of the light industry supply chain. For example, supply chain node association rules define the cooperative relationship and dependency relationship between upstream and downstream enterprises, logistics process constraint rules standardize the time window and path selection of cargo transportation, resource allocation rules constrain the rational allocation of resources such as manpower, equipment, and funds, and business logic rules ensure that supply chain operations comply with industry standards and corporate policies. This systematic rule base construction not only improves the business rationality of the analysis results, but also provides strong prior knowledge support for the algorithm, avoiding the analysis results that may not conform to business logic that may occur in pure data-driven methods.

[0041] The knowledge rule matching algorithm designed in step S4.2 realizes the intelligent integration of data-driven discovery and expert knowledge verification. The algorithm can accurately match the cross-modal association patterns discovered in step S3 with domain knowledge rules, and automatically identify which associations conform to the supply chain business logic and which may be false associations caused by data noise or anomalies. For example, if data analysis finds a strong correlation between a supplier's negative review text and high-quality product images, the matching algorithm will judge the rationality of this correlation based on business logic rules, and may identify that this is due to evaluation lags, incorrect evaluation objects, or other business anomalies. This intelligent matching mechanism significantly improves the credibility and business applicability of multimodal analysis results, avoiding the black box problem that may arise from traditional machine learning methods.

[0042] The rule constraint function constructed in step S4.3 realizes the regulation of cross-modal association relationships, and carries out targeted processing of different types of association relationships through differentiated weight setting strategies. For valid association relationships that conform to business logic, the system sets enhanced weights to strengthen their influence in subsequent analysis; for abnormal association relationships that do not conform to business logic, the system sets suppression weights to reduce their negative impact. This two-way adjustment mechanism can not only highlight valuable association information, but also effectively suppress noise interference, significantly improving the accuracy and reliability of the analysis results. For example, when the system finds a positive correlation between increased production efficiency and an increase in equipment failure reports, the rule constraint function will judge the irrationality of this association based on business logic and set suppression weights to reduce the impact of this false association.

[0043] The adaptive weight adjustment mechanism in step S4.4 not only considers the static constraints of the rule-constraint weight matrix but also incorporates the dynamic changes in current supply chain status information, adjusting the importance weights of each modal feature based on real-time business conditions. For example, during normal supply chain operation, text and image modalities may dominate; however, in emergencies or abnormal situations, audio and video modalities may become more important because they can provide more timely on-site information. This dynamic adjustment capability enables the method to flexibly adapt to different scenarios and stages of supply chain management, improving its practicality and adaptability.

[0044] The element-by-element weight fusion operation employed in step S4.5 enables refined control of multimodal features, ensuring that domain knowledge constraints can be effectively implemented at the feature level. By performing element-by-element multiplication of the modal feature weight adjustment matrix with the original eigenvector, this method enables business-logic-based feature importance adjustments while maintaining the structural integrity of the eigenvector. This fusion operation avoids the potential information loss that can result from coarse-grained weight adjustments, ensuring that the adjusted eigenvectors both conform to business logic constraints and retain their original rich information content.

[0045] The feature consistency verification mechanism in step S4.6 provides important quality assurance for the entire weight adjustment process. This verification mechanism can detect whether the feature vector after weight adjustment still maintains the integrity of the original semantic information, preventing excessive weight adjustment from causing distortion or loss of feature information. By setting reasonable consistency thresholds and verification standards, the mechanism can ensure that the adjusted feature vector not only meets business logic constraints but also retains sufficient information value for subsequent fusion analysis. This dual guarantee strategy significantly improves the robustness and reliability of the method, laying a solid foundation for subsequent attention fusion and decision analysis.

[0046] Example 3: Different from Example 2, the specific steps of step S5 are as follows: S5.1. Receive the weight-adjusted multimodal feature vector set outputted in step S4, including the weight-adjusted text feature vector, image feature vector, acoustic feature vector, and video feature vector; S5.2. Construct a cross-modal attention fusion network, which includes a query vector generation module, a key-value vector generation module, and an attention weight calculation module; S5.3. Input the weighted multimodal feature vector set into the query vector generation module and key-value vector generation module of the cross-modal attention fusion network to generate corresponding query vector matrices and key-value vector matrices, respectively; S5.4. Calculate a cross-modal attention weight matrix based on the query vector matrix and the key-value vector matrix through the attention weight calculation module. The cross-modal attention weight matrix represents the importance relationship between feature vectors of different modalities. S5.5. Based on the cross-modal attention weight matrix, perform a weighted fusion operation on the weight-adjusted multimodal feature vector set to achieve interactive fusion of cross-modal feature information; S5.6. Perform dimensionality transformation and nonlinear mapping on the fused feature representation through a fully connected neural network layer to generate a unified fused feature vector of fixed dimension; S5.7. Standardize the unified fusion feature vector to ensure the numerical stability of the feature vector, and output the standardized unified fusion feature vector as input data for step S6.

[0047] The overall design of step S5 fully considers the special needs and challenges of supply chain decision analysis. Supply chain management involves multiple stakeholders, diverse business scenarios, and a dynamically changing market environment. The multimodal information that needs to be fused and processed is highly complex and uncertain. This method, through a carefully designed attention fusion network, can intelligently identify and integrate the information most important for the current decision-making task, while suppressing the influence of irrelevant or interfering information. This targeted design makes the fused features generated by this method highly valuable in decision support, providing high-quality feature input for subsequent supply chain status assessment and optimized configuration. For example, when conducting supplier evaluations, the system can intelligently integrate the supplier's historical delivery records (text), product quality images, audio recordings of communication and coordination, and on-site audit videos to generate comprehensive supplier evaluation features, providing a scientific basis for supplier selection decisions.

[0048] Specifically, the cross-modal attention fusion network constructed in step S5.2 adopts a modular architectural design. Through the division of labor and cooperation among the query vector generation module, the key-value vector generation module, and the attention weight calculation module, efficient and flexible cross-modal information fusion is achieved. This modular design has multiple advantages: first, each module has clear functions and responsibilities, which facilitates system maintenance and functional expansion; second, the standardized interface design between modules makes the system scalable, and specific modules can be adjusted or replaced as needed; finally, the modular architecture facilitates parallel computing and performance optimization, which can significantly improve the efficiency of large-scale data processing. In actual supply chain management applications, this architectural design can flexibly adapt to the needs of multimodal data fusion of different scales and complexities, providing enterprises with scalable technical solutions.

[0049] In steps S5.3-S5.4, the self-attention mechanism of the traditional Transformer architecture is extended to cross-modal scenarios, enabling intelligent assessment of the importance of inter-modal information. The query vector matrix encodes the information requirements of the current analysis task, and the key-value vector matrix encodes the information content of each modal feature. By calculating the similarity between the query and the key value, an attention weight matrix is ​​generated. This matrix can accurately quantify the importance of different modal features in a specific analysis task. For example, when conducting a supply chain risk assessment, the system may automatically increase its attention to abnormal sounds (audio modality) and images of equipment failures (image modality); when conducting a customer satisfaction analysis, the system will pay more attention to customer feedback text and service quality videos. This intelligent attention allocation mechanism significantly improves the targetedness and accuracy of multimodal fusion.

[0050] The weighted fusion operation based on the attention weight matrix in step S5.5 realizes the true interactive fusion of cross-modal features, rather than simple feature splicing or average fusion. This mechanism dynamically adjusts the contribution weights of different modal features so that the final fusion result can integrate the advantageous information of each modality while suppressing the influence of redundant or interfering information. The advantages of this deep interactive fusion mechanism are: on the one hand, it can discover and utilize complementary information between different modalities to improve the integrity of the overall information; on the other hand, it can adaptively balance the contributions of different modalities according to specific task requirements and improve the task relevance of the fusion results. In supply chain management, this mechanism can effectively integrate diverse information from different data sources and provide more comprehensive and accurate feature representation for decision analysis.

[0051] Step S5.6 performs dimensionality transformation and nonlinear mapping on the fused features through a fully connected neural network layer, significantly enhancing the expressive power and generalization performance of the feature representation. The dimensionality transformation operation ensures that the fused feature vector has a fixed dimensional specification, which facilitates the standardization of subsequent models; the nonlinear mapping operation introduces nonlinear transformation through the activation function, so that the fused features can express more complex inter-modal relationships and business logic. The technical value of this processing strategy lies in: it not only maintains the rich information content of multimodal fusion, but also enhances the abstract expression ability of features through nonlinear transformation, so that the fused features can better adapt to complex supply chain decision-making tasks. For example, the original multimodal features may only be able to express linear correlation relationships. After nonlinear mapping, the fused features can capture complex nonlinear business laws and potential patterns in the supply chain.

[0052] The standardization processing mechanism in step S5.7 provides an important guarantee for numerical stability of the entire fusion process, ensuring the reliability and consistency of the fused feature vector in subsequent processing. In the complex multimodal fusion process, due to factors such as differences in the numerical range of different modal data and cumulative errors in calculation accuracy, the fusion result may be numerically unstable. This standardization mechanism eliminates these potential numerical problems through normalization processing, ensuring that the fused feature vector has good numerical characteristics. This processing strategy is particularly important for application scenarios such as supply chain management, which have extremely high requirements for accuracy and reliability. It can effectively avoid decision-making errors or system instability caused by numerical problems.

[0053] In this embodiment, the specific steps of step S6 are: S6.1. Receive the standardized unified fusion feature vector output from step S5 and construct a supply chain decision analysis model. The supply chain decision analysis model includes a supply chain status assessment module and a supply chain optimization decision module. S6.2. Input the standardized unified fusion feature vector into the supply chain status assessment module, analyze the current supply chain operation status through a multi-layer neural network, and identify the operating efficiency, resource utilization, and risk level of each link in the supply chain; S6.3. Generate a supply chain status assessment report based on the analysis results of the supply chain status assessment module. The supply chain status assessment report includes overall supply chain health indicators, performance indicators of each node, and potential risk warning information; S6.4. Input the standardized unified fusion feature vector and the supply chain status assessment report into the supply chain optimization decision module, and calculate the optimal supply chain configuration parameters through the deep neural network; S6.5. Generate a supply chain optimization plan based on the optimal supply chain configuration parameters. The supply chain optimization plan includes resource allocation adjustment suggestions, process optimization strategies, and risk control measures. S6.6. Format and output the supply chain status assessment report and supply chain optimization plan to form a complete supply chain analysis and decision-making result, and provide it to the light industry supply chain management system for decision-making execution.

[0054] In step S6.1, the status assessment module is responsible for comprehensively diagnosing the current supply chain's operational status and identifying problems and opportunities. The optimization decision-making module, based on these diagnostic results, proposes specific improvement plans and configuration adjustments. The advantage of this architectural design lies in decomposing complex supply chain decision-making problems into two relatively independent yet interrelated sub-problems, reducing system complexity while improving the scientific nature and operability of decision-making. For example, when the system detects a delivery delay from a supplier, the status assessment module quantifies the impact of this delay on the overall supply chain. Based on this assessment, the optimization decision-making module proposes specific solutions, such as activating alternative suppliers, adjusting inventory buffers, or rescheduling delivery plans.

[0055] The supply chain health assessment function implemented in steps S6.2-S6.3 uses a multi-layer neural network to comprehensively analyze three key dimensions: operational efficiency, resource utilization, and risk level, establishing a three-dimensional supply chain health monitoring system. Operational efficiency analysis focuses on time efficiency and process smoothness across each link, identifying efficiency issues such as production bottlenecks and logistics delays. Resource utilization analysis assesses the rationality and efficiency of resource allocation, including human resources, equipment, and capital, and identifies instances of resource waste or shortages. Risk level analysis predicts the probability and impact of potential risks such as supply disruptions, quality issues, and cost fluctuations. For example, in a textile company's supply chain analysis, the system might discover that production efficiency is 85% (good), but raw material inventory turnover is only 60% (low). Furthermore, the system detects a medium policy risk level in the region where the main supplier is located. The system then integrates this information to generate a health assessment report containing specific values, trend analysis, and early warning alerts.

[0056] The supply chain optimization decision module in steps S6.4-S6.5 uses deep neural networks to achieve an intelligent transformation from problem diagnosis to solution generation. This module not only processes the comprehensive information provided by a single fused feature vector, but also combines the detailed analysis results in the status assessment report to calculate the optimal supply chain configuration parameters through a complex optimization algorithm. This dual-input design ensures that the optimization decision is based on both the original multimodal information and the structured evaluation results, improving the comprehensiveness and accuracy of the decision. For example, when the system finds that the capacity utilization rate of a production base is too low, the optimization decision module will comprehensively consider the base's equipment status (image information), employee satisfaction (text feedback), production noise level (audio information), and operational process efficiency (video analysis), and then generate a comprehensive optimization plan that includes equipment update recommendations, personnel training plans, process improvements, and performance incentives.

[0057] The supply chain optimization plan generated in step S6.5 utilizes a structured output format encompassing three dimensions: resource allocation adjustment recommendations, process optimization strategies, and risk control measures. This ensures the comprehensiveness and actionability of the decision-making recommendations. Resource allocation adjustment recommendations provide specific guidance on the reallocation of human, material, and financial resources; process optimization strategies focus on improving business processes and enhancing efficiency, including suggestions for operational standardization and automation upgrades; and risk control measures provide preventative and response strategies for identified potential risks. This structured output facilitates understanding and implementation by business managers and provides a standardized framework for subsequent decision tracking and effectiveness evaluation.

[0058] Taking the daily necessities production supply chain of a light industrial enterprise as an example, the specific operation process of the system is as follows: Condition Assessment Phase: System analysis revealed abnormal peaks in the sound spectrum of equipment operating in Production Workshop A (audio modality). Simultaneously, product quality inspection images revealed surface defects in a batch of products (image modality). Quality inspection reports documented the issues (text modality). On-site surveillance video revealed irregularities in key process steps (video modality). Through multimodal fusion analysis, the system identified a complex issue involving equipment maintenance, quality control, and personnel training. It then quantified the impact on the overall supply chain: operational efficiency decreased by 15%, resource utilization dropped to 78%, and the quality risk level was raised to high.

[0059] Optimization decision-making stage: Based on the status assessment results, the system generates specific optimization plans: (1) Resource allocation adjustment suggestions include arranging professional technicians to maintain and repair abnormal equipment, deploying quality inspection personnel to strengthen the comprehensive inspection of the batch of products, and temporarily deploying skilled operators from other production lines for support; (2) Process optimization strategies include revising the standard operating procedures of the process, adding real-time monitoring nodes for the equipment's operating status, and establishing a rapid response mechanism for quality issues; (3) Risk control measures include establishing an equipment preventive maintenance plan, strengthening operator skills training and assessment, and improving the multi-level inspection system for quality control.

[0060] Decision execution support: The system integrates these analysis results and optimization suggestions into standardized decision reports, including problem diagnosis summaries, impact assessments, solution details, and implementation timelines, and provides them to the enterprise management system for decision execution and effect tracking.

[0061] In this embodiment, the multimodal data collection in step S1 specifically includes: Collect text data from the light industrial supply chain environment through the Internet of Things sensor network. The text data includes production order information, logistics document information, quality inspection reports, and supplier communication records. Collect image data in the light industrial supply chain environment through industrial camera equipment. The image data includes production site images, product quality images, warehouse status images, and logistics transportation images. Audio data from the light industrial supply chain environment is collected through sound collection equipment. The audio data includes the operating sounds of production equipment, logistics vehicles, and operating environment noise information. Video data in the light industrial supply chain environment is collected through video surveillance equipment. The video data includes production process videos, logistics operation videos and security monitoring videos.

[0062] The technical advantage of collecting text data through IoT sensor networks lies in its ability to automate, real-time, and standardize the acquisition of text data, avoiding the errors and delays that can result from manual data entry. The automated collection of production order information ensures real-time synchronization and accurate execution of production plans; the digital collection of logistics document information enables full tracking of the goods flow process; the electronic collection of quality inspection reports ensures the timeliness and accuracy of quality data; and the systematic collection of supplier communication records facilitates subsequent relationship management and performance evaluation. For example, at a furniture manufacturing company, the IoT system can automatically capture the entire chain of text information, from raw material purchase orders to finished product delivery confirmation, including supplier quotations, production process sheets, quality inspection certificates, logistics delivery notes, etc., forming a complete data chain and providing rich structured and semi-structured text data for subsequent analysis.

[0063] Multi-scene image acquisition by industrial cameras enables visual monitoring and data recording throughout the entire supply chain. Production site images reflect the real-time status of the production line, equipment operation, and operating environment conditions. Product quality images provide a visual record of key quality information, such as product appearance, dimensions, and defects. Warehouse status images record important information such as inventory levels, cargo placement, and the warehouse environment. Logistics and transportation images capture key logistics processes, such as cargo loading and unloading, transportation processes, and delivery status. This comprehensive visual data collection not only provides an objective basis for quality control and process monitoring but also lays the data foundation for intelligent analysis based on computer vision. For example, product surface images captured by high-resolution industrial cameras can be automatically identified through deep learning algorithms, with detection accuracy often exceeding that of manual visual inspection. Warehouse status images, using image recognition technology, automatically count inventory quantities and assess storage conditions, significantly improving the accuracy and efficiency of inventory management.

[0064] The classified collection of different types of audio data by sound collection equipment demonstrates a deep dive into the value of acoustic information within the supply chain environment. The collection of operating sounds from production equipment provides crucial data support for equipment health monitoring and predictive maintenance. Sound spectrum analysis can detect equipment anomalies and signs of failure early on. Monitoring the operating sounds of logistics vehicles helps assess the working status and efficiency of transportation equipment. The collection of operating environment noise information not only focuses on occupational health and safety compliance but also reflects the intensity and pace of production activities. This multi-dimensional collection of audio information provides a unique data perspective for intelligent supply chain management. For example, a precision machinery manufacturer, through the deployment of high-precision acoustic sensors, is able to detect minute abnormal vibrations in machine tool spindle bearings, predicting bearing failures 3-5 days in advance. This avoids production interruptions and quality issues caused by sudden failures, significantly improving equipment reliability and production continuity.

[0065] The multi-scene dynamic recording capabilities of video surveillance equipment enable continuous monitoring and complete recording of key supply chain processes. Production process videos can fully record the entire production operation, providing valuable information for process optimization, quality traceability, and training and education. Logistics operation videos record key operational links such as cargo handling, loading and unloading, and sorting, helping to identify operational irregularities and efficiency bottlenecks. Security surveillance videos not only ensure operational safety but also provide objective evidence for accident analysis and prevention. The temporal and continuous nature of video data makes it a valuable data source for analyzing dynamic processes and trends. For example, by analyzing production process videos, it is possible to identify differences in operator proficiency in specific steps and uncover subtle issues that affect production efficiency. By analyzing logistics operation videos, cargo handling routes and loading and unloading processes can be optimized, improving logistics operational efficiency.

[0066] The acquisition of data in four different modalities enables more comprehensive, accurate, and real-time access to supply chain information. Text data provides structured business information and decision-making basis; image data provides intuitive status information and quality evidence; audio data provides real-time feedback on equipment status and environmental conditions; and video data provides a complete record of process dynamics and operational details. This collaborative collection of multimodal data not only improves the integrity and reliability of information but also provides a rich data foundation for subsequent multimodal fusion analysis. For example, when a quality problem arises on a production line, the system can simultaneously capture quality inspection reports (text), images of product defects (images), abnormal equipment sounds (audio), and videos of the operation process (video). Through cross-validation and comprehensive analysis of multimodal information, the root cause of the problem can be more accurately identified and solutions formulated.

[0067] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A light industry supply chain multimodal data fusion analysis method based on deep learning, characterized by: The following steps are included: S1. Collect text data, image data, audio data, and video data in a light industry supply chain environment to form a multimodal raw data set, and perform data cleaning, noise filtering, and format standardization on the multimodal raw data set to obtain a normalized multimodal data set; S2. For the text data, image data, audio data, and video data in the normalized multimodal dataset, respectively, a text feature extraction network, an image feature extraction network, an audio feature extraction network, and a video feature extraction network are used to perform feature extraction to generate corresponding text feature vectors, image feature vectors, audio feature vectors, and video feature vectors; S3. Using a cross-modal correlation analysis algorithm, calculate the correlation coefficients among the text feature vector, the image feature vector, the audio feature vector, and the video feature vector, and construct a cross-modal feature correlation matrix; S4. Based on the pre-built light industry supply chain domain knowledge rule base, domain knowledge constraints are applied to the cross-modal feature correlation matrix, and the weight coefficients of each modal feature vector are dynamically adjusted to obtain a set of multimodal feature vectors after weight adjustment. S5. Construct a cross-modal attention fusion network, take the weight-adjusted multimodal feature vector set as input, calculate the attention weights between the feature vectors of each modality through the self-attention mechanism, realize the interactive fusion of cross-modal feature information, and output a unified fusion feature vector; S6. Build a supply chain decision analysis model, use the unified fusion feature vector as input feature, generate supply chain status evaluation results and supply chain optimization configuration parameters through neural network mapping, and finally generate a supply chain optimization plan.

2. The method for multimodal data fusion analysis of light industry supply chain based on deep learning according to claim 1 is characterized by: The specific steps of step S2 are: S2.

1. Performing a spatiotemporal alignment operation on the text data, image data, audio data, and video data in the normalized multimodal dataset. By extracting the timestamp information and spatial coordinate information of each modal data, a spatiotemporal mapping relationship between the multimodal data is established. Different modal data describing the same supply chain event are correlated and matched to form a spatiotemporally aligned multimodal dataset. S2.

2. For the text data in the spatiotemporally aligned multimodal dataset, a BERT pre-trained language model based on the Transformer architecture is used to extract global semantic features. The text sequence information is processed through a bidirectional long short-term memory network to capture the contextual semantic dependencies in the supply chain text data and generate a fixed-dimensional text feature vector. S2.

3. For the image data in the spatiotemporally aligned multimodal dataset, a ResNet-50 deep convolutional neural network architecture is used to extract multi-scale visual features of the images through a residual connection mechanism. The edge, texture, shape, and spatial layout feature information of the supply chain scene images are extracted layer by layer to generate corresponding image feature vectors. S2.

4. Extracting spectral features of the audio data in the spatiotemporally aligned multimodal dataset using a one-dimensional convolutional neural network. Inputting the spectral features into a long short-term memory network to process audio temporal dependencies, identify sound pattern features in the supply chain environment, and generate acoustic feature vectors representing the audio information. S2.

5. For the video data in the spatiotemporally aligned multimodal dataset, use a three-dimensional convolutional neural network to simultaneously extract the spatial and temporal features of the video, combine it with a multi-head self-attention mechanism to identify key action sequences in the supply chain operation process, and generate a video feature vector containing spatiotemporal information; S2.

6. Perform L2 norm normalization on each of the text feature vector, image feature vector, acoustic feature vector, and video feature vector to normalize the modulus of each modal feature vector to unit length, thereby eliminating differences in numerical dimensions between feature vectors of different modalities. S2.

7. Combining the normalized modal feature vectors to form a multimodal feature vector set, which serves as input data for the cross-modal feature correlation analysis in step S3.

3. The method for multimodal data fusion analysis of light industry supply chain based on deep learning according to claim 2 is characterized by: The specific steps of step S3 are: S3.

1. Receive the multimodal feature vector set, construct a cross-modal feature pair combination, and combine the feature vectors of any two different modalities to form a binary feature pair, forming a total of N(N-1) / 2 feature pairs, where N is the total number of modalities; S3.

2. Perform a dimensional alignment operation on the binary feature pairs, map the feature vectors of different dimensions to a unified dimensional space through a fully connected neural network layer, calculate the linear correlation coefficient of each feature pair using a canonical correlation analysis algorithm, and construct a linear correlation coefficient matrix; S3.

3. Construct a cross-modal semantic encoding network, perform semantic space mapping on the binary feature pairs, train a semantic encoder by maximizing the similarity of different modal features of the same supply chain event and minimizing the similarity of different modal features of different supply chain events; calculate the semantic similarity between feature vectors based on the trained semantic encoder, and construct a semantic similarity matrix; S3.

4. Based on the linear correlation coefficient matrix and the semantic similarity matrix, calculate the high-order nonlinear correlation relationship between cross-modal features through a deep neural network to generate a high-order correlation matrix; S3.

5. Assign weight coefficients α, β, and γ to the linear correlation coefficient matrix, semantic similarity matrix, and high-order relevance matrix, where α + β + γ = 1; adaptively learn the weight coefficients through a weight optimization network, and calculate a comprehensive relevance index using a weighted linear combination; S3.

6. Construct an N×N dimensional cross-modal feature correlation matrix based on the comprehensive correlation index, where the matrix elements represent the correlation strength between different modalities and the diagonal elements represent the autocorrelation strength within the modalities; S3.

7. Perform standardization on the cross-modal feature correlation matrix, use the minimum-maximum normalization method to map the correlation value to the [0,1] interval, and perform noise correlation suppression to generate a standardized cross-modal feature correlation matrix as input data for step S4.

4. The method for multimodal data fusion analysis of a light industry supply chain based on deep learning according to claim 3 is characterized by: The specific steps of step S4 are: S4.

1. Receive the standardized cross-modal feature correlation matrix output from step S3 and call a pre-built light industry supply chain domain knowledge rule base, which includes supply chain node association rules, logistics process constraint rules, resource allocation rules, and business logic rules. S4.

2. Design a knowledge rule matching algorithm to match the association patterns in the standardized cross-modal feature correlation matrix with various rules in the pre-built light industry supply chain domain knowledge rule library, identifying valid associations that conform to the supply chain business logic and marking abnormal associations that do not conform to the business logic. S4.

3. Based on the knowledge rule matching results, a rule constraint function is constructed to set suppression weights for relationships marked as abnormal, and enhancement weights for relationships that conform to business logic, thereby generating a rule constraint weight matrix. S4.

4. Design an adaptive weight adjustment mechanism to dynamically calculate the importance weight coefficients of the text feature vector, image feature vector, acoustic feature vector, and video feature vector based on the rule constraint weight matrix and the current supply chain status information, and generate a modal feature weight adjustment matrix; S4.

5. Performing an element-by-element weight fusion operation on the modal feature weight adjustment matrix and each modal feature vector after normalization output from step S2 to achieve dynamic adjustment of feature weights based on the constraints of the pre-built light industry supply chain domain knowledge rule base; S4.

6. Perform feature consistency verification on the text feature vector, image feature vector, acoustic feature vector, and video feature vector after weight adjustment to ensure that the adjusted feature vector maintains the integrity of the original semantic information, and obtain the multimodal feature vector set after weight adjustment as the input data of step S5.

5. The method for multimodal data fusion analysis of light industry supply chain based on deep learning according to claim 4 is characterized by: The specific steps of step S5 are: S5.

1. Receive the weight-adjusted multimodal feature vector set outputted in step S4, including the weight-adjusted text feature vector, image feature vector, acoustic feature vector, and video feature vector; S5.

2. Construct a cross-modal attention fusion network, which includes a query vector generation module, a key-value vector generation module, and an attention weight calculation module; S5.

3. Input the weight-adjusted multimodal feature vector set into the query vector generation module and key-value vector generation module of the cross-modal attention fusion network to generate corresponding query vector matrices and key-value vector matrices, respectively; S5.

4. Calculating, by an attention weight calculation module, a cross-modal attention weight matrix based on the query vector matrix and the key-value vector matrix, where the cross-modal attention weight matrix represents the importance relationship between feature vectors of different modalities; S5.

5. Based on the cross-modal attention weight matrix, perform a weighted fusion operation on the weight-adjusted multimodal feature vector set to achieve interactive fusion of cross-modal feature information; S5.

6. Perform dimensionality transformation and nonlinear mapping on the fused feature representation through a fully connected neural network layer to generate a unified fused feature vector of fixed dimension; S5.

7. Standardize the unified fusion feature vector to ensure the numerical stability of the feature vector, and output the standardized unified fusion feature vector as input data for step S6.

6. The method for multimodal data fusion analysis of light industry supply chain based on deep learning according to claim 5 is characterized by: The specific steps of step S6 are: S6.

1. Receive the standardized unified fusion feature vector output from step S5 and construct a supply chain decision analysis model. The supply chain decision analysis model includes a supply chain status assessment module and a supply chain optimization decision module. S6.

2. Input the standardized unified fusion feature vector into the supply chain status assessment module, analyze the current supply chain operation status through a multi-layer neural network, and identify the operating efficiency, resource utilization, and risk level of each link in the supply chain; S6.

3. Generate a supply chain status assessment report based on the analysis results of the supply chain status assessment module. The supply chain status assessment report includes overall supply chain health indicators, performance indicators of each node, and potential risk warning information; S6.

4. Input the standardized unified fusion feature vector and the supply chain status assessment report into the supply chain optimization decision module, and calculate the optimal supply chain configuration parameters using a deep neural network; S6.

5. Generate a supply chain optimization plan based on the optimal supply chain configuration parameters. The supply chain optimization plan includes resource allocation adjustment suggestions, process optimization strategies, and risk control measures. S6.

6. Format and output the supply chain status assessment report and the supply chain optimization plan to form a complete supply chain analysis and decision-making result, which is provided to the light industry supply chain management system for decision-making execution.

7. The method for multimodal data fusion analysis of light industry supply chain based on deep learning according to claim 1 is characterized by: The multimodal data acquisition in step S1 specifically includes: Collect text data from the light industrial supply chain environment through the Internet of Things sensor network, including production order information, logistics document information, quality inspection reports, and supplier communication records; Collect image data in a light industrial supply chain environment using industrial camera equipment, including production site images, product quality images, storage status images, and logistics transportation images; Collecting audio data from the light industry supply chain environment using sound collection equipment, including the operating sounds of production equipment, logistics vehicles, and operating environment noise information; Video data in a light industry supply chain environment is collected through video surveillance equipment, and the video data includes production process videos, logistics operation videos, and security monitoring videos.

Images