Multi-mode enterprise intelligent brain system based on artificial intelligence

Abstract

The invention discloses a multi-mode enterprise intelligent brain system based on artificial intelligence, and belongs to the technical field of big data processing. The method and the device are used for solving the technical problems of poor multi-modal data processing precision and poor system robustness in the existing scheme. Semantic alignment of multi-modal features can be realized through a closed-loop process of pre-extraction, comparative learning, dimension alignment, dynamic optimization and verification and fine adjustment, cross-modal semantic association is established through comparative learning, space mismatching is solved through dimension alignment, dynamic matrix optimization fits task requirements, and actual effects are ensured through verification and fine adjustment. Through a progressive process of confidence evaluation, task coding, weight calculation, feature fusion and end-to-end optimization, intelligent weighting of multi-modal features is realized, and through confidence evaluation, reliable data is screened, a decision target is determined through task coding, a gating network dynamically allocates weights, and end-to-end optimization ensures that a strategy meets service requirements.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and more specifically to a multimodal enterprise intelligent brain system based on artificial intelligence. Background Technology

[0002] Multimodal Enterprise Brain is a comprehensive solution that combines artificial intelligence technology, aiming to provide enterprises with all-round capability enhancements such as decision support, process optimization, and customer service. By integrating data processing capabilities in multiple forms such as text, images, audio, and video, and utilizing technologies such as machine learning, deep learning, and natural language processing, it enables in-depth analysis and understanding of internal and external enterprise data.

[0003] The quality and distribution of enterprise data often fluctuate with business scenarios. The fusion weights of existing technical solutions are mostly fixed values ​​trained offline, which cannot be adjusted according to the confidence level of real-time data. For example, when the noise of a certain modality data surges, it is still assigned a high weight. In addition, during the multimodal data acquisition process, equipment failure, environmental interference or human operation errors can all introduce noise. Existing technologies mostly use single-modal anomaly detection. For example, the isolated forest algorithm is only used for structured data and ignores the correlation anomalies of cross-modal data. Summary of the Invention

[0004] The purpose of this invention is to provide a multimodal enterprise intelligent brain system based on artificial intelligence to solve the technical problems of poor accuracy and robustness of multimodal data processing in existing solutions.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] The AI-based multimodal enterprise intelligence system includes a multimodal data acquisition module that acquires multimodal data from enterprise operations in real time. The multimodal data includes text commands, voice interaction records, business process images, structured business data, and equipment sensor data.

[0007] The adaptive feature alignment module extracts modal features corresponding to multimodal data based on a multimodal pre-trained model and aligns the heterogeneous feature semantic space through a dynamic attention mechanism.

[0008] The dynamic fusion decision module uses a gated recurrent unit combined with a Bayesian probability model to dynamically adjust the fusion weights of each modality based on the data confidence level and output a multimodal joint representation.

[0009] The robustness enhancement module generates perturbation data samples through adversarial training, combines them with anomaly detection models to identify data noise or attack behavior, and triggers feature reconstruction or policy rollback mechanisms.

[0010] Preferably, the implementation steps of the adaptive feature alignment module include extracting and initially normalizing the modal features corresponding to the multimodal data to obtain the initially normalized features corresponding to different modal data.

[0011] Layer normalization is performed on each modal feature to obtain the preliminary normalized features corresponding to each modal feature.

[0012] Preferably, when performing cross-modal contrastive learning and constructing semantic associations, positive sample pairs and negative sample pairs are constructed respectively, and the similarity of positive sample pairs is maximized while the similarity of negative sample pairs is minimized.

[0013] A modality-specific embedding layer is introduced to perform dimensionality normalization on text, image, and speech features respectively, thereby eliminating the feature space differences between modalities.

[0014] Design a learnable alignment matrix and optimize the matrix parameters through backpropagation to achieve semantic alignment of heterogeneous feature vectors.

[0015] Preferably, construct positive sample pairs Where k is the sample index;

[0016] Construct negative sample pairs Where l≠k, m≠k;

[0017] Define the contrastive loss function L contrast To maximize the similarity of positive sample pairs and minimize the similarity of sample pairs:

[0018]

[0019] In the formula, |P| represents the number of positive sample pairs; cos() represents the cosine similarity, cos(a,b) represents the similarity between the feature vectors of anchor sample a and positive sample b; cos(a,c) represents the similarity between the feature vectors of anchor sample a and negative sample c; exp(cos(a,b) / τ) represents the matching score of the positive sample pair; τ is a temperature parameter used to control the smoothness of the distribution, τ>0.

[0020] Preferably, a modality-specific embedding layer is introduced to perform dimensionality normalization on text, image, and speech features respectively, eliminating differences in feature space between modalities; specifically:

[0021] For each modality, a learnable linear transformation layer is designed to map the initially normalized features to a unified dimension d. align Among them, the text embedding layer:

[0022]

[0023] Image embedding layer:

[0024] Voice embedding layer:

[0025]

[0026] For the mapped embedded features f t f i f s Perform L2 normalization to ensure that the embedded features are distributed on a unit sphere.

[0027] Preferably, a learnable alignment matrix M∈R is designed. dalign×dalign embedding features f of each modality t f i f s Mapping to the shared semantic space z t =M·f t z i =M·f i z s =M·f s ;

[0028] Using the cross-modal retrieval task as the supervision signal, the retrieval loss L is defined. retrieval Optimize the parameters of the alignment matrix M:

[0029] L retrieval =CrossEntropy(Sim(z t , z i )y)+CrossEntropy(Sim(z t , z s ), y);

[0030] Where Sim() is the cosine similarity, y is the binary label, 1 indicates a match, and 0 indicates a mismatch.

[0031] Preferably, the implementation steps of the dynamic fusion decision module include calculating the confidence score of each modality feature. The confidence score is output by a pre-trained confidence evaluation model, and the evaluation indicators include feature entropy value and cosine similarity with historical data.

[0032] Construct a gating network, input the confidence scores of each modality and the current task type, and output the fusion weight coefficients of each modality;

[0033] Multimodal features are fused using a weighted summation method to generate a joint representation vector that is relevant to the task.

[0034] Preferably, the aligned multimodal features are input into a pre-trained confidence evaluation model, and the confidence score of each modality is output.

[0035] The confidence score is calculated using the following formula: In the formula, H(z) is the characteristic entropy; The cosine similarity is represented by z; z represents the feature vector of the current sample to be analyzed. α represents the average feature of historical data of the same type; α∈[0,1] is the weighting coefficient of feature entropy and cosine similarity.

[0036] Preferably, the robustness enhancement module includes an anomaly detection submodule, a feature reconstruction unit, and a policy fallback unit; the anomaly detection submodule, based on the isolated forest algorithm or variational autoencoder, performs anomaly scoring on multimodal features, and marks them as anomalous data when the score exceeds a threshold;

[0037] The feature reconstruction unit, for abnormal data, calls normal features from similar historical scenes for interpolation reconstruction, or generates alternative features through a generative adversarial network;

[0038] The strategy rollback unit automatically switches to the rule-based traditional decision-making mode and triggers a manual review process when a high proportion of abnormal data is detected continuously.

[0039] Compared to existing solutions, the beneficial effects achieved by this invention are:

[0040] This invention achieves semantic alignment of multimodal features through a closed-loop process of pre-extraction, contrastive learning, dimensional alignment, dynamic optimization, and verification fine-tuning. Contrastive learning establishes cross-modal semantic associations, dimensional alignment resolves spatial mismatches, dynamic matrix optimization aligns with task requirements, and verification fine-tuning ensures practical effectiveness. Each step is interconnected, ultimately improving the accuracy of multimodal fusion and the system's ability to handle heterogeneous data. Furthermore, a progressive process of confidence assessment, task encoding, weight calculation, feature fusion, and end-to-end optimization enables intelligent weighting of multimodal features. Confidence assessment filters reliable data, task encoding clarifies decision-making objectives, and gating networks... Dynamic weight allocation and end-to-end optimization ensure that strategies align with business needs. Each step is interconnected, ultimately improving the accuracy of multimodal fusion and the robustness of decision-making. This allows the enterprise's intelligent brain system to flexibly adjust data utilization strategies according to actual scenarios. Through a full-process design of anomaly detection, feature reconstruction, and strategy rollback, a closed-loop mechanism of detection, repair, and fallback is formed. This enables anomaly detection to quickly locate problems, feature reconstruction to reduce the impact of data loss, and strategy rollback to ensure business continuity. The close correlation between each step ultimately allows the enterprise's intelligent brain system to maintain high-precision decision-making even in scenarios with data noise, attacks, or anomalies, thus improving system stability. Attached Figure Description

[0041] The invention will now be further described with reference to the accompanying drawings.

[0042] Figure 1 This is a flowchart illustrating the operation of the multimodal enterprise intelligent brain system based on artificial intelligence according to the present invention. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figure 1 As shown, the present invention is a multimodal enterprise intelligent brain system based on artificial intelligence, including a multimodal data acquisition module, an adaptive feature alignment module, a dynamic fusion decision module, and a robustness enhancement module;

[0045] The multimodal data acquisition module acquires multimodal data from enterprise operations in real time. Multimodal data includes text commands, voice interaction records, business process images, structured business data, and equipment sensor data.

[0046] Among them, text instructions mainly come from non-private chat records of internal enterprise collaboration tools, user requirement descriptions in the work order system, business instructions in the email system, etc. By calling the open APIs of collaboration tools, such as DingTalk robot API and Lark message push API, text data is pulled or subscribed to in real time using HTTP / REST protocol. The collection frequency can be configured to real time or event trigger.

[0047] Voice interaction recordings: voice files or streaming data from scenarios such as customer service hotlines and conference recordings; for stored voice files, such as call recordings from customer service systems, they are periodically pulled from storage servers of Alibaba Cloud OSS and / or enterprise NAS via FTP / SFTP protocols; for real-time voice streams, such as telephone conferences, they are accessed through the voice gateway via RTMP / RTSP protocols, transcoded into PCM / MP3 format in real time, and transmitted to the acquisition module.

[0048] Business process images: including monitoring images of the production workshop, such as production line quality inspection photos; scanned copies of contracts / invoices, such as financial reimbursement forms; real-time monitoring images captured by enterprise camera devices, or image files uploaded through the document management system, which are imported in batches using the HTTP file upload interface;

[0049] Structured business data: database tables from enterprise ERP, CRM, OA and other systems, such as sales orders, inventory records and employee attendance; direct connection to the database via JDBC / ODBC interface, or periodic extraction of incremental data via ETL tools to ensure data synchronization with business systems; ETL tools, specifically Apache NiFi;

[0050] Equipment sensor data: Sensors from production equipment and logistics warehousing equipment, such as temperature, pressure, and displacement sensors; accessed via industrial IoT protocols such as MQTT and OPC UA, sensor data is pushed to the acquisition module in JSON format at fixed intervals;

[0051] The collected data is checked for completeness and consistency based on a preset rule base, and data with incorrect format or missing fields is automatically filtered out. The completeness and consistency checks are existing conventional technical solutions, and the specific implementation steps are not described here.

[0052] In this embodiment of the invention, the multimodal data acquisition module can realize the transformation from scattered heterogeneous data to a unified feature vector, thereby providing high-quality and standardized input data support for subsequent operation mode matching and scheduling optimization.

[0053] The adaptive feature alignment module extracts modal features corresponding to multimodal data based on a multimodal pre-trained model and aligns heterogeneous feature semantic spaces through a dynamic attention mechanism; the specific steps include:

[0054] A cross-modal contrastive learning framework is constructed, which is trained using positive and negative samples to maximize the mutual information of cross-modal features;

[0055] Among them, positive samples are different modal data of the same content;

[0056] Negative samples are modal data with different content; specifically:

[0057] Modal features corresponding to multimodal data are extracted and preliminarily normalized to obtain preliminarily normalized features corresponding to different modal data.

[0058] For text data, a pre-trained BERT model, such as RoBERTa, is used to extract word-level features, and the output of the [CLS] tag is taken as the global feature vector of the text. d t t represents the text feature dimension; t represents the text modality.

[0059] For image data, the ResNet-50 model is used to extract global pooling features from the output of convolutional layers. d i is the image feature dimension; i represents the image modality;

[0060] For the speech data, the Wav2Vec 2.0 model is used to extract frame-level features, which are then aggregated into global speech features through a self-attention layer. d s Let s represent the speech feature dimension, and s represent the speech modality.

[0061] Layer normalization is performed on each modal feature to eliminate dimensional differences between different modal features, resulting in preliminary normalized features for each modal feature.

[0062] In this embodiment of the invention, a pre-trained model is used to extract features from multimodal data. Prior knowledge from large-scale data is used to enhance the semantic expressive power of the features. Layer normalization is used to ensure that the numerical distribution of features of different modalities is consistent, providing stable and reliable input data support for subsequent alignment.

[0063] When performing cross-modal contrastive learning and constructing semantic associations, positive sample pairs and negative sample pairs are constructed respectively, and the similarity of positive sample pairs is maximized while the similarity of negative sample pairs is minimized.

[0064] Specifically, construct positive sample pairs Where k is the sample index, such as text-image or text-speech pairs with the same content;

[0065] Construct negative sample pairs Where l≠k, m≠k, l is the global index of the current positive sample, m is the modality type identifier, and m∈{t,i,s};

[0066] Define the contrastive loss function L contrast To maximize the similarity of positive sample pairs and minimize the similarity of sample pairs:

[0067]

[0068] In the formula, |P| represents the number of positive sample pairs; cos() represents the cosine similarity, cos(a,b) represents the similarity between the feature vectors of anchor sample a and positive sample b; cos(a,c) represents the similarity between the feature vectors of anchor sample a and negative sample c; exp(cos(a,b) / τ) represents the matching score of the positive sample pair; τ is a temperature parameter used to control the smoothness of the distribution, τ>0;

[0069] It should be noted that by contrastive learning, cross-modal features are forced to be close to features with the same content and far away from features with different content in the semantic space, thus establishing cross-modal semantic associations. The temperature parameter can be used to adjust the sensitivity of the loss function to similarity, avoiding training instability caused by excessive feature differences.

[0070] A modality-specific embedding layer is introduced to perform dimensionality normalization on text, image, and speech features respectively, eliminating feature space differences between modalities; specifically:

[0071] For each modality, a learnable linear transformation layer, such as an embedding layer, is designed to map the initially normalized features to a uniform dimension d. align ;

[0072] Among them, the text embedding layer:

[0073]

[0074] Image embedding layer:

[0075] Voice embedding layer:

[0076]

[0077] For the mapped embedded features f t f i f s Perform L2 normalization to ensure that the embedded features are distributed on a unit sphere;

[0078] It should be noted that linear transformation compresses the high-dimensional features of heterogeneous modalities to a uniform dimension, solving the problem of feature space mismatch between different modalities; L2 normalization makes the embedded features more evenly distributed in the semantic space, avoiding similarity calculation errors caused by modulus differences.

[0079] Design a learnable alignment matrix and optimize the matrix parameters through backpropagation to achieve semantic alignment of heterogeneous feature vectors;

[0080] Among them, a learnable alignment matrix M∈R is designed. dalign×dalign embedding features f of each modality t f i f s Mapping to the shared semantic space z t =M·f t z i =M·f i z s =M·f s ;

[0081] Using cross-modal retrieval tasks as the supervision signal, such as recalling matching images / speech given a text query, the retrieval loss L is defined. retrieval Optimize the parameters of the alignment matrix M:

[0082] L rerieval =CrossEntropy(Sim(z t ,z i ),y)+CrossEntropy(Sim(z t , z s ), y);

[0083] Where Sim() is the cosine similarity, y is the binary label, 1 indicates a match, and 0 indicates a mismatch;

[0084] It should be noted that by using the dynamic alignment matrix M, the mapping direction of each modality feature can be flexibly adjusted, making the features in the shared semantic space more suitable for downstream tasks such as retrieval and inference; the retrieval loss is directly related to the actual application scenario, which can enhance the adaptability of the aligned features to business tasks.

[0085] When performing dynamic verification and parameter fine-tuning of alignment effects, cross-modal alignment metrics, such as recall rate for text-image retrieval, are calculated on the validation set.

[0086] If the metric is less than the corresponding preset threshold, the pre-trained model parameters are frozen, and only the embedding layer W is fine-tuned. t W i W s and alignment matrix M;

[0087] If the metric is greater than or equal to the corresponding preset threshold, the current parameters are retained and deployed to the production environment. Meanwhile, new data is continuously collected during the online learning phase, and the above-mentioned incremental training is triggered periodically.

[0088] It should be noted that the verification process ensures that the aligned features meet actual business needs, avoiding ineffective optimization for the sake of alignment; and through online incremental training, the system can adapt to dynamic changes in enterprise data, such as new business terms and new image formats, maintaining long-term robustness.

[0089] In this embodiment of the invention, a closed-loop process of pre-extraction-contrast learning-dimensional alignment-dynamic optimization-verification fine-tuning can be used to achieve semantic alignment of multimodal features. Contrast learning establishes cross-modal semantic associations, dimensional alignment solves spatial mismatches, dynamic matrix optimization fits task requirements, and verification fine-tuning ensures practical results. Each step is interconnected, ultimately improving the accuracy of multimodal fusion and the system's ability to process heterogeneous data.

[0090] The dynamic fusion decision module employs a gated recurrent unit combined with a Bayesian probability model to dynamically adjust the fusion weights of each modality based on data confidence, outputting a multimodal joint representation. Specific steps include:

[0091] Calculate the confidence score for each modality feature. The confidence score is output by a pre-trained confidence evaluation model. Evaluation metrics include feature entropy value and cosine similarity with historical data.

[0092] Among them, the aligned multimodal features z t z i z s Input each modality into the pre-trained confidence evaluation model and output the confidence score c for each modality. t c i c s The values ​​are all in the range of [0,1].

[0093] The confidence assessment model takes a feature vector as input and calculates its output through a fully connected layer and a sigmoid activation function. Its loss function incorporates the following metrics:

[0094] Feature entropy is z d The value of the d-th dimension of the feature vector is given. The smaller the feature entropy, the more concentrated the feature distribution and the higher the confidence level.

[0095] Cosine similarity to historical data z represents the feature vector of the current sample to be analyzed. The average feature of similar historical data is used; the higher the similarity, the more consistent it is with known patterns, and the higher the confidence level.

[0096] Final confidence score α∈[0,1] is the weighting coefficient of feature entropy and cosine similarity, which is 0.5 by default;

[0097] It should be noted that by using both entropy and historical similarity as indicators to assess confidence, we can capture the stability of the feature itself and measure its matching degree with known business patterns, thus avoiding fusion bias caused by noise in single-modal data such as blurry images and noisy speech.

[0098] Construct a gating network, input the confidence scores of each modality and the current task type, and output the fusion weight coefficients of each modality; specifically,

[0099] Encode the current business task type as a one-hot vector t∈{0,1} K K represents the total number of task types, such as 5 types; business task types, such as "risk assessment", "demand forecasting", and "customer profiling";

[0100] The task code t′ is compared with the confidence scores of each modality c. t c i c s The concatenation of these vectors forms the input vector x = [t′; c] of the gating network. t c i c s ]; where “;” indicates vector concatenation;

[0101] The gated network employs a two-layer fully connected structure: the first layer maps x to the hidden layer h = ReLU(W1·x+b1); where W1∈R h×(K+3 b1∈R h h represents the dimension of the hidden layer, for example, 64.

[0102] The second layer outputs the fusion weights for each modality, w = Softmax(W2·h + b2); where W2 ∈ R. 3×h b2∈R3 ;w = [w t w i w s ], satisfying w t +w i +w s =1;

[0103] It should be noted that task type encoding enables the gating network to perceive the current decision-making objective. For example, risk assessment relies more on text contract terms, while demand forecasting relies more on structured sales data. After combining confidence scores, the network can dynamically adjust weights to adapt to task requirements. The Softmax output ensures that the weights are non-negative and sum to 1, avoiding unreasonable negative weights or weight imbalances.

[0104] Multimodal features are fused using a weighted summation method to generate a task-related joint representation vector; specifically:

[0105] Based on the weights w output by the gating network, the aligned multimodal features z are... t z i z s Perform a weighted summation to generate the joint representation z. fusion :z fusion =w t ·z t +w i ·z i +w s ·z s ;

[0106] For joint characterization z fusion Perform layer normalization processing and output to downstream decision modules, such as risk classifiers and demand forecasting models;

[0107] Through task loss L tast And weighted smoothing loss L smooth Define the overall loss function L total :L total =L task +λ·L smooth Among them, the task loss L tast : Cross-entropy loss or mean squared error loss of downstream tasks;

[0108] Weighted smoothing loss In the formula, T is the total number of time steps, and w t Let be the weight vector at step t, and λ be the smoothing loss weight, which is 0.1 by default.

[0109] The parameters W1, b1, W2, b2 of the gated network and the parameters of the confidence evaluation model are optimized through backpropagation.

[0110] It should be noted that by directly linking task losses to business objectives, the effectiveness of the fused representation for decision-making is ensured; and by using weighted smoothing loss to constrain the magnitude of weight changes between adjacent time steps, decision jumps caused by data fluctuations can be prevented, which can effectively enhance system stability.

[0111] In this embodiment of the invention, a progressive process of confidence assessment, task coding, weight calculation, feature fusion, and end-to-end optimization is adopted to achieve intelligent weighting of multimodal features. Confidence assessment filters reliable data, task coding clarifies decision-making objectives, gating networks dynamically allocate weights, and end-to-end optimization ensures that the strategy aligns with business needs. Each step is interconnected, ultimately improving the accuracy of multimodal fusion and the robustness of decision-making, enabling the enterprise's intelligent brain system to flexibly adjust data utilization strategies according to actual scenarios.

[0112] The robustness enhancement module generates perturbation data samples through adversarial training, combines them with an anomaly detection model to identify data noise or attack behavior, and triggers feature reconstruction or policy rollback mechanisms.

[0113] The robustness enhancement module includes an anomaly detection submodule, a feature reconstruction unit, and a policy fallback unit.

[0114] The anomaly detection submodule, based on the Isolation Forest algorithm or Variational Autoencoder, scores multimodal features for anomalies, marking data as anomalous when the score exceeds a threshold; specifically:

[0115] The aligned joint representation z fusion With the original features z of each modality t z i z s Concatenate into a high-dimensional feature vector Where, d total =d align ×4, including joint representation and 3 types of single-modal features;

[0116] If there are sufficient data labels, i.e., normal / abnormal samples are known, a variational autoencoder (VAE) is used to model the normal data distribution:

[0117] The VAE maps the high-dimensional feature vector X to the latent space z through the encoder q(z|X), and the decoder p(X|z) reconstructs the original data. The loss function is:

[0118] L VAE =-E q(z|X) [log p(X|z)]+KL(q(z|X)||p(z));

[0119] Among them, E q(z|X)[log p(X|z)] is the expectation term, which calculates the expectation of the latent variable z under the encoder distribution q(z|x) and calculates the average value of the decoder likelihood logp(x|z), which is used to measure the difference between the original data and the reconstructed data. The second term is the KL divergence, which is used to constrain the latent space distribution to be close to the prior latent variable p(z), which is usually a standard normal distribution.

[0120] If data labels are missing, the Isolation Forest algorithm is used: multiple isolated trees are constructed by randomly partitioning the feature space. Abnormal samples have shorter paths within the trees, and the anomaly score s(X) is calculated using the following formula: In the formula, h(X) is the path length of the sample in the isolated tree, E(h(X)) is the average path length of multiple trees, c(n) is the correction factor, n is the number of samples, and s(X)∈[0,1] indicates that the closer to 1, the more abnormal it is;

[0121] The abnormal score s is output according to the model type and compared with the preset threshold θ. When s≥θ, it is marked as abnormal data; where θ can be 0.8.

[0122] It should be noted that by using the isolated forest algorithm and variational autoencoder in combination for different data label situations, the anomaly detection needs of different data scenarios can be covered, thereby effectively improving the diversity and reliability of anomaly detection.

[0123] The feature reconstruction unit, for abnormal data, calls normal features from similar historical scenes for interpolation reconstruction, or generates alternative features through a generative adversarial network;

[0124] Regarding the abnormal data X abn Search the historical normal database for the k most similar normal samples and use cosine similarity. Sort the data, j = 1, 2, 3, ..., k; j represents different normal samples;

[0125] Features of the k matched normal samples Substitute features X are generated using similarity-weighted linear interpolation. rec :

[0126] If historical data is insufficient, a Generative Adversarial Network (GAN) can be trained using normal data as input, with the generator G outputting surrogate features X that are consistent with the distribution of normal data. gan Discriminator D distinguishes between real and generated data, and the loss function is:

[0127] L GAN =E X～pdata [log D(X)]+E z～pz [log(1-D(G(z)))];

[0128] In the formula, D(X) is the output of the discriminator D on the real sample x, with a value range of [0,1], representing the probability that X is real data; z is the noise vector sampled from the prior noise distribution; G(z) is the fake data sample converted by the generator G from the noise z; D(G(z)) is the output of the discriminator D on the generated sample G(z), representing the probability that G(z) is real data; pdata is the real data distribution, and pz is the noise distribution input to the generator, such as a normal distribution; E X～pdata E represents the expectation of the actual data distribution. z～pz This represents the expectation of the noise distribution;

[0129] It should be noted that using normal data from similar historical scenarios for interpolation and reconstruction can preserve the continuity of business logic; for example, when production equipment malfunctions, normal data from recent equipment of the same model can be used to fill the gap; GAN-based reconstruction is suitable for small sample scenarios, expanding the distribution of normal data through generative models to avoid decision failures caused by missing data.

[0130] The strategy rollback unit automatically switches to a rule-based traditional decision-making mode and triggers a manual review process when a high proportion of abnormal data is continuously detected. Specifically:

[0131] The proportion of outlier data within T consecutive time steps is calculated as r = number of outlier samples / T; where T can be 10.

[0132] If r ≥ δ, the system automatically switches to the rule-driven backup decision mode; where δ is the switching threshold, which can be 0.5.

[0133] Decision results are generated based on the enterprise’s pre-defined business rules, avoiding reliance on unreliable multimodal fusion features; among these, business rules include triggering replenishment when inventory is below a safe value and marking high risk when there are more than 3 customer complaints.

[0134] After the switch is triggered, the system automatically sends an anomaly report to the administrator. The anomaly report includes anomaly data samples, anomaly detection scores, and reconstruction attempt results. After review, the administrator can select:

[0135] If the anomaly is confirmed to be a real business risk, such as a system attack, update the threshold of the anomaly detection model or refactor the strategy.

[0136] Additionally, if a false detection is identified, the parameters of the anomaly detection model are adjusted, for example, by lowering the threshold θ.

[0137] It should be noted that in extreme and abnormal scenarios, such as when the system suffers a malicious data attack or a large-scale sensor failure, the strategy rollback unit ensures the basic functions of the system through rule patterns and avoids business interruption due to model failure. The manual review closed loop realizes machine detection + manual calibration, which can effectively improve the system's adaptability to complex anomalies.

[0138] In this embodiment of the invention, a closed-loop mechanism of detection, repair, and fallback is formed through the full-process design of anomaly detection, feature reconstruction, and policy rollback. This enables anomaly detection to quickly locate problems, feature reconstruction to reduce the impact of data loss, and policy rollback to ensure business continuity. The steps are closely related, ultimately enabling the enterprise's intelligent brain system to maintain high-precision decision-making even in the face of data noise, attacks, or anomaly scenarios, thereby improving the system's stability.

[0139] In the several embodiments provided by this invention, it should be understood that the disclosed system can be implemented in other ways. For example, the embodiments of the invention described above are merely illustrative; for example, the division of modules is only a logical functional division, and there may be other division methods in actual implementation.

[0140] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0142] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the essential characteristics of the present invention.

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multimodal enterprise intelligent brain system based on artificial intelligence, characterized in that: It includes a multimodal data acquisition module to acquire multimodal data in enterprise operations in real time. The multimodal data includes text commands, voice interaction records, business process images, structured business data, and equipment sensor data. The adaptive feature alignment module extracts modal features corresponding to multimodal data based on a multimodal pre-trained model and aligns the heterogeneous feature semantic space through a dynamic attention mechanism. The dynamic fusion decision module uses a gated recurrent unit combined with a Bayesian probability model to dynamically adjust the fusion weights of each modality based on the data confidence level and output a multimodal joint representation. The robustness enhancement module generates perturbation data samples through adversarial training, combines them with anomaly detection models to identify data noise or attack behavior, and triggers feature reconstruction or policy rollback mechanisms.

2. The multimodal enterprise intelligent brain system based on artificial intelligence according to claim 1, characterized in that, The implementation steps of the adaptive feature alignment module include extracting modal features corresponding to multimodal data and performing preliminary normalization to obtain preliminary normalized features corresponding to different modal data. Layer normalization is performed on each modal feature to obtain the preliminary normalized features corresponding to each modal feature.

3. The multimodal enterprise intelligent brain system based on artificial intelligence according to claim 2, characterized in that, When performing cross-modal contrastive learning and constructing semantic associations, positive sample pairs and negative sample pairs are constructed respectively, and the similarity of positive sample pairs is maximized while the similarity of negative sample pairs is minimized. A modality-specific embedding layer is introduced to perform dimensionality normalization on text, image, and speech features respectively, thereby eliminating the feature space differences between modalities. Design a learnable alignment matrix and optimize the matrix parameters through backpropagation to achieve semantic alignment of heterogeneous feature vectors.

4. The multimodal enterprise intelligent brain system based on artificial intelligence according to claim 3, characterized in that, Construct positive sample pairs Where k is the sample index; Construct negative sample pairs Where l≠k, m≠k; Define the contrastive loss function L contrast To maximize the similarity of positive sample pairs and minimize the similarity of sample pairs: In the formula, |P| represents the number of positive sample pairs; cos() represents the cosine similarity, cos(a,b) represents the similarity between the feature vectors of anchor sample a and positive sample b; cos(a,c) represents the similarity between the feature vectors of anchor sample a and negative sample c; exp(cos(a,b) / τ) represents the matching score of the positive sample pair; τ is a temperature parameter used to control the smoothness of the distribution, τ>0.

5. The multimodal enterprise intelligent brain system based on artificial intelligence according to claim 3, characterized in that, A modality-specific embedding layer is introduced to perform dimensionality normalization on text, image, and speech features respectively, eliminating feature space differences between modalities; specifically: For each modality, a learnable linear transformation layer is designed to map the initially normalized features to a unified dimension d. align Among them, the text embedding layer: Image embedding layer: Voice embedding layer: For the mapped embedded features f t f i f s Perform L2 normalization to ensure that the embedded features are distributed on a unit sphere.

6. The multimodal enterprise intelligent brain system based on artificial intelligence according to claim 3, characterized in that, Design a learnable alignment matrix M∈R dalign×dalign embedding features f of each modality t f i f s Mapping to the shared semantic space z t =M·f t z i =M·f i z s =M·f s ; Using the cross-modal retrieval task as the supervision signal, the retrieval loss L is defined. retrieval Optimize the parameters of the alignment matrix M: L rerievat =CrossEntropy(Sim(z t ,from i ),y)+CrossEntropy(Sim(z t ,from s ),y); Where Sim() is the cosine similarity, y is the binary label, 1 indicates a match, and 0 indicates a mismatch.

7. The multimodal enterprise intelligent brain system based on artificial intelligence according to claim 3, characterized in that, The implementation steps of the dynamic fusion decision module include calculating the confidence score of each modality feature. The confidence score is output by a pre-trained confidence evaluation model. The evaluation metrics include feature entropy value and cosine similarity with historical data. Construct a gating network, input the confidence scores of each modality and the current task type, and output the fusion weight coefficients of each modality; Multimodal features are fused using a weighted summation method to generate a joint representation vector that is relevant to the task.

8. The multimodal enterprise intelligent brain system based on artificial intelligence according to claim 7, characterized in that, The aligned multimodal features are input into the pre-trained confidence evaluation model, and the confidence score of each modality is output. The confidence score is calculated using the following formula: In the formula, H(z) is the characteristic entropy; The cosine similarity is represented by z; z represents the feature vector of the current sample to be analyzed. α represents the average feature of similar historical data; α∈[0,1] is the weighting coefficient of feature entropy and cosine similarity.

9. The multimodal enterprise intelligent brain system based on artificial intelligence according to claim 1, characterized in that, The robustness enhancement module includes an anomaly detection submodule, a feature reconstruction unit, and a policy fallback unit. The anomaly detection submodule, based on the isolated forest algorithm or variational autoencoder, scores multimodal features for anomalies and marks them as anomalous data when the scores exceed a threshold. The feature reconstruction unit, for abnormal data, calls normal features from similar historical scenes for interpolation reconstruction, or generates alternative features through a generative adversarial network; The strategy rollback unit automatically switches to the rule-based traditional decision-making mode and triggers a manual review process when a high proportion of abnormal data is detected continuously.

Images