Intelligent business decision support system based on multi-modal data and knowledge graph

Abstract

The invention discloses an intelligent business decision support system based on multi-modal data and a knowledge graph, and relates to the technical field of data analysis. In order to solve the problems of weak data integration capability and poor model adaptability in existing business analysis, the adopted scheme comprises a multi-modal data processing module used for collecting multi-modal data and constructing a knowledge graph after preprocessing and feature extraction; the model training and optimizing module is used for taking a large-scale pre-training language model as a core, acquiring external knowledge from the atlas in combination with a retrieval enhancement generation technology, performing parameter fine tuning, optimizing a training process and generating an optimized model at the same time; the intelligent decision generation module is used for generating a preliminary suggestion based on the optimization model and generating a report through logic deviation evaluation and correction; the dynamic feedback module is used for constructing evaluation system quantitative model performance by taking the report as input and combining real-time business data; and meanwhile, user feedback is collected to form an incremental sample dynamic optimization model. The accuracy of business decision making can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data analysis, in particular to an intelligent business decision support system based on multi-modal data and knowledge graph. BACKGROUND

[0002] In the field of business decision-making, data-driven intelligent analysis systems are crucial for enterprises to gain insights into market trends and optimize strategies. Traditional methods rely on single-modal data (such as structured tables), which makes it difficult to integrate multi-source heterogeneous data such as text and images, leading to one-sided analysis. For example, decision-making models based solely on sales table data cannot fully utilize the potential information in user reviews, advertisement images, and other unstructured data, limiting the comprehensiveness of decision-making.

[0003] In the prior art, deep learning models face the problem of small sample overfitting when processing table data, and lack effective processing capabilities for unstructured data. At the same time, although large language models (LLM) perform well in natural language processing, their "black box" nature leads to opaque decision-making logic, making it difficult to meet the strict requirements of explainability in fields such as finance and compliance. In addition, traditional systems rely on static reports and fixed model parameters, which cannot respond to market changes and dynamically adjust to user needs in real time, resulting in insufficient model adaptability. SUMMARY

[0004] The present application provides an intelligent business decision support system based on multi-modal data and knowledge graph to address the weak integration of heterogeneous data, poor model adaptability, and low transparency of decision-making in existing business analysis.

[0005] The intelligent business decision support system based on multi-modal data and knowledge graph of the present application adopts the following technical solutions to solve the above technical problems:

[0006] An intelligent business decision support system based on multi-modal data and knowledge graph, comprising:

[0007] A multi-modal data processing module for collecting three types of multi-modal data: text, table, and image. After differential preprocessing and feature extraction, a knowledge graph is constructed based on an entity-relation model to support dynamic updates in the time dimension, achieving real-time semantic mapping of business dynamics.

[0008] A model training and optimization module for using a large-scale pre-trained language model as the core, receiving the dynamic knowledge graph output by the multi-modal data processing module as the basic data, and combining retrieval enhancement generation technology to obtain external knowledge. At the same time, low-rank matrix decomposition technology is used for parameter lightweight fine-tuning, and adaptive learning rate and multi-task loss function are used to optimize the training process to generate an optimized model that adapts to the business scenario.

[0009] An intelligent decision generation module is configured to generate preliminary decision suggestions and verify logical rationality by relying on an optimization model output by the model training and optimization module and combining dynamic knowledge graphs and real-time business data provided by the multi-modal data processing module, to evaluate logical bias in the suggestions through feature importance analysis, and to generate an explainability report including a reasoning process and visualized basis by correcting the suggestions by fusing real-time data and domain rules.

[0010] A dynamic feedback module is configured to construct a multi-dimensional evaluation system quantitative model performance by taking the explainability report output by the intelligent decision generation module as a core input and combining real-time business data of the multi-modal data processing module; meanwhile, the dynamic feedback module is configured to form incremental samples by collecting user feedback, to realize a closed-loop iteration of "data-decision-feedback-optimization" by inversely acting on the model training and optimization module through a reinforcement learning algorithm based on a policy gradient.

[0011] Optionally, the multi-modal data processing module specifically includes:

[0012] A multi-modal data acquisition unit is configured to acquire three types of multi-modal data including text, table and image.

[0013] A differentiated preprocessing unit is configured to start a differentiated preprocessing procedure according to characteristics of different types of data: to perform semantic analysis preprocessing on text data to lay a foundation for sentiment analysis and entity recognition; to perform structure conversion on table data to adapt to low-dimensional vector conversion requirements; and to extract visual key information from image data through target detection technology to prepare for structured conversion and ensure that various types of data are converted into effective formats that can be used for feature analysis.

[0014] A multi-modal feature extraction unit is configured to perform deep feature extraction on the preprocessed multi-modal data, specifically including: for text data, performing sentiment analysis and named entity recognition by using a deep learning model based on an attention mechanism to extract semantic features of user behavior keywords and sentiment tendencies; for table data, adopting a visual Transformer transfer learning architecture to capture inter-row and inter-column correlation through a hierarchical attention mechanism, to convert table data into low-dimensional embedding vectors, to support multi-dimensional feature aggregation of time, geography and product category, and to realize dynamic aggregation analysis of sales data by quarter and region; and for image data, using a convolutional neural network combined with a YOLO series target detection technology to identify key information including price labels and promotion signs in product images, and then converting visual information into structured data through OCR technology to supplement entity details for the knowledge graph.

[0015] A knowledge graph construction unit is configured to extract three types of entities including "products", "markets" and "consumers", and associated relationships including "sales-geographical-growth" and "user-preference-product", from multi-modal features based on an entity-relation model, to construct a knowledge graph.

[0016] The knowledge graph updating unit is configured to learn semantic association between entities by using a graph neural network algorithm, introduce a time dimension weight, and dynamically adjust edge weight of the graph according to data update frequency, so that the knowledge graph can map business dynamic changes in real time.

[0017] Further, the knowledge graph updating unit dynamically adjusts the edge weight of the graph by the following formula to realize real-time mapping of the knowledge graph to business dynamics:

[0018] ω e→e` = σ1*Cooccurrence(e, e') + σ2*Sentiment(e, e'),

[0019] wherein ω e→e represents the weight of the edge between the entity e and the entity e' in the knowledge graph; σ1 and σ2 are weight coefficients for adjusting the influence degree of the co-occurrence frequency and the text sentiment intensity of the entities on the edge weight; and σ1 + σ2 = 1; Cooccurrence(e, e') represents the co-occurrence frequency of the entities e and e', and Sentiment(e, e') represents the text sentiment intensity between the entities e and e', so as to realize real-time mapping of the knowledge graph to business dynamics.

[0020] Optionally, the model training and optimization module specifically includes:

[0021] The core model adaptation unit is configured to take a large-scale pre-trained language model as a basic engine, rely on a retrieval enhancement generation technology to retrieve the dynamic knowledge graph output by the multi-modal data processing module to obtain external knowledge, filter redundant information by a semantic matching algorithm, generate context input fused with multi-source knowledge, and enhance the understanding ability of the model to complex business problems; and introduce a low-rank matrix decomposition technology to perform model lightweight fine-tuning, decompose a high-dimensional parameter matrix into a low-rank sub-matrix, and realize optimization of key parameters of the model.

[0022] The model intelligent training unit is configured to dynamically adjust a learning rate according to the complexity of the data output by the multi-modal data processing module, realize stable training in a complex data scenario by an adaptive learning rate formula, and improve the convergence efficiency of the model; and jointly optimize intent classification, SQL query generation, and decision interpretability tasks, adopt a multi-task loss function, and ensure that the model optimizes logical transparency while generating a decision result.

[0023] Further, the adaptive learning rate formula is as follows:

[0024]

[0025] wherein η tThe adaptive learning rate at time t, i.e., the learning rate adopted by the model when training to the t-th step; η0 represents the initial learning rate, which is the basic learning rate set by the model at the beginning of training, i.e., t=0; λ represents an adjustment coefficient, which is used to control the influence degree of data complexity on the degree of learning rate decay, and can be adjusted according to actual needs or experimental results; DataComplexity(t) represents the data complexity at time t, which is used to quantify the complexity of the current training data, and its value will change with the change of the data in the training process.

[0026] Further optionally, the calculation formula of the multi-task loss function is as follows:

[0027] L = α1*L intent + α2*L sql + α3*L explain ,

[0028] In the formula, L represents the total loss of the multi-task, which is the overall loss obtained by weighted combination of the task losses of intent classification, SQL query generation, and decision explainability, and is used to guide the parameter update of the model in the multi-task joint training process; L intent represents the loss of the intent classification task, which is used to measure the difference between the prediction result and the true result of the model in the intent classification task; L sql represents the loss of the SQL query generation task, which is used to evaluate the accuracy of the SQL query generated by the model, and reflects the gap between the performance of the model in this task and the expectation; L explain represents the loss of the decision explainability task, which is used to reflect the quality of the decision explanation generated by the model, and is calculated based on the local explainable model-independent explanation technology, and is used to optimize the logical transparency of the model decision; α1, α2, α3 respectively represent the weight coefficients of the task losses of intent classification, SQL query generation, and decision explainability, which are used to adjust the importance of different tasks in the total loss, and can be set according to the needs of each task or experimental results.

[0029] Optionally, the intelligent decision generation module specifically comprises:

[0030] The suggestion generation and verification unit is used to generate a preliminary decision suggestion for a business problem based on the optimized model output by the model training and optimization module and the dynamic knowledge graph constructed by the multi-modal data processing module, and simultaneously call a business rule engine to verify the rationality of the preliminary suggestion against the basic business logic;

[0031] The suggestion evaluation unit is used to identify logical bias in the preliminary suggestion based on real-time business data provided by the multi-modal data processing module, detect whether there is an analysis blind area in the regional dimension and the time dimension by quantifying the contribution of each input feature to the decision result, and troubleshoot possible logical vulnerabilities in the preliminary suggestion.

[0032] A correction and report generation unit is configured to correct the preliminary suggestion based on the real-time market data provided by the multi-modal data processing module and the preset field rules, and generate an explainability report, present the decision logic tree through a visualization tool, and mark the reasoning path corresponding to the knowledge graph of the multi-modal data processing module, so as to improve the transparency and credibility of the decision.

[0033] Further, the involved suggestion evaluation unit specifically adopts a Shapley value algorithm to identify the logical bias in the preliminary suggestion, and the formula is as follows:

[0034]

[0035] In the formula, Φ i represents the Shapley value of the i-th feature, which is used to quantify the average marginal contribution of the i-th feature to the decision result; N represents a set of all features, and S represents a subset selected from the set N; represents that the subset S is selected from “all features except the i-th feature”; |S| represents the number of features contained in the subset S; n represents the total number of features, i.e., the size of the set N; v(S) represents the decision value of the model when using the subset S; and v(S∪{i}) represents the decision value corresponding to the case where the feature i is added to the subset S.

[0036] Optionally, the involved dynamic feedback module specifically includes:

[0037] An evaluation system construction unit is configured to take the explainability report output by the intelligent decision generation module as a core input, combine the real-time business data provided by the multi-modal data processing module, and construct a multi-dimensional evaluation system covering accuracy, coverage, diversity, and explainability; subsequently, the evaluation system is used to calculate the values of each index in real time through a statistical learning method, so as to provide a quantitative basis for model performance evaluation; wherein, the accuracy measures the matching degree between the decision result and the actual business data output by the multi-modal data processing module, the coverage reflects the comprehensiveness of the model in calling multi-modal data dimensions, the diversity reflects the differentiation level of the suggestion scheme, and the explainability quantifies the clarity of the decision logic through the logical transparency in the explainability report;

[0038] A feedback data acquisition unit is configured to acquire feedback data of the user on the output result of the intelligent decision generation module, including the change of the business index monitored by the multi-modal data processing module after the strategy is implemented, the correction record of the artificial on the explainability report and the decision suggestion, process the acquired feedback data into an incremental training sample, and provide data support for subsequent model optimization;

[0039] The reinforcement learning optimization unit is used to adopt a policy gradient-based reinforcement learning algorithm to link the model training and optimization module for full-process optimization: firstly, the indexes of the multi-dimensional evaluation system are integrated into the reward function, and the pros and cons of the output results of the intelligent decision-making generation module are quantified through the reward function; secondly, the incremental training samples are input, and the real-time data updating learning process of the multi-modal data processing module is combined to guide the model training and optimization module to adjust the pre-training model parameters and optimize the training strategy; finally, the model decision-making ability is continuously updated to form a closed-loop iterative process of "data-decision-feedback-optimization", ensuring that the full-link module adapts to the business demand iteration.

[0040] Further optionally, the expression of the reward function is:

[0041]

[0042] In the formula, R total represents the cumulative total reward, which is the total reward obtained by the model in the entire time range from time 0 to time T, integrating the performance of all aspects; t represents the time step, used to identify different time points for reward calculation, advancing from the initial time 0 to the final time T; T is the total time step, that is, the length of time during which the entire decision or interaction process lasts, ending at time T; γ is a discount factor, taking a value between 0 and 1, used to measure the importance of future rewards in the present; is a weight coefficient, respectively used to adjust the contribution degree of R acc , R exp , R div in the total reward; R acc represents the accuracy reward, used to measure the matching degree of the model decision result and the actual data; R exp represents the coverage rate reward, used to reflect the comprehensiveness of the data dimensions involved in the model; R div represents the diversity reward, used to reflect the differentiation degree of the model generated proposal scheme.

[0043] The intelligent business decision-making support system based on multi-modal data and knowledge graph of the present application has the following beneficial effects compared with the prior art:

[0044] The present application can effectively improve the comprehensiveness, accuracy and transparency of business decision-making, solve the problems of weak heterogeneous data integration capability, poor model adaptability and low decision-making transparency in existing business analysis; it is suitable for complex scenarios such as finance, retail, e-commerce, etc., and provides efficient and interpretable intelligent decision-making support for enterprises, and has significant commercial application value. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings illustrate the present application. Figure 1 is the module connection block diagram of the present application. DETAILED DESCRIPTION

[0046] To make the technical solution, the technical problem solved, and the technical effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments.

[0047] Example 1:

[0048] Reference Appendix Figure 1 This embodiment proposes an intelligent business decision support system based on multimodal data and knowledge graphs, which includes a multimodal data processing module, a model training and optimization module, and an intelligent decision generation module.

[0049] The multimodal data processing module is used to collect three types of multimodal data: text, tables, and images. After differentiated preprocessing and feature extraction, it constructs a knowledge graph that supports dynamic updates over time based on an entity-relationship model, enabling real-time semantic mapping of business dynamics.

[0050] In this embodiment, the multimodal data processing module specifically includes:

[0051] The multimodal data acquisition unit is used to acquire three types of multimodal data: text, tables, and images, providing raw data input for subsequent processing.

[0052] The differentiated preprocessing unit is used to initiate differentiated preprocessing processes based on the characteristics of different types of data: semantic parsing preprocessing is performed on text data to lay the foundation for sentiment analysis and entity recognition; structural transformation is performed on tabular data to adapt it to the requirements of low-dimensional vector transformation; and visual key information is extracted from image data through object detection technology to prepare for structured transformation, ensuring that all types of data are transformed into an effective format that can be used for feature analysis.

[0053] The multimodal feature extraction unit is used to perform deep feature extraction on preprocessed multimodal data. Specifically, it includes: for text data, using deep learning models based on attention mechanisms (such as bidirectional encoder networks) to perform sentiment analysis and named entity recognition, extracting semantic features such as user behavior keywords and sentiment tendencies (such as positive / negative polarity labels); for tabular data, using a visual Transformer transfer learning architecture, capturing the relationships between rows and columns through a hierarchical attention mechanism, converting tabular data into low-dimensional embedding vectors, supporting multi-dimensional feature aggregation of time, geography, and product category, and realizing dynamic aggregation analysis of sales data by quarter and region; for image data, using convolutional neural networks combined with YOLO series object detection technology to identify key information such as price tags and promotional logos in product images, and then using OCR technology to convert visual information into structured data, supplementing entity details for the knowledge graph;

[0054] The knowledge graph construction unit is configured to extract three types of entities, i.e., products, markets, and consumers, and the associated relationships, i.e., sales-geographical growth and user-preference-product, from the multi-modal features based on an entity-relation model, and construct a knowledge graph.

[0055] The knowledge graph updating unit is configured to learn semantic associations between entities by using a graph neural network algorithm, introduce a time dimension weight, and dynamically adjust the edge weight of the graph based on the data update frequency, so that the knowledge graph can map the business dynamic changes in real time.

[0056] The knowledge graph updating unit dynamically adjusts the edge weight of the graph by the following formula, so as to realize real-time mapping of the knowledge graph to the business dynamic changes:

[0057] ω e→e` =σ1*Cooccurrence(e,e`)+σ2*Sentiment(e,e`),

[0058] In the formula, ω e→e represents the weight of the edge between entity e and entity e` in the knowledge graph; σ1 and σ2 are weight coefficients for adjusting the influence degree of the co-occurrence frequency and the text sentiment intensity of the entities on the edge weight; and σ1+σ2=1; Cooccurrence(e,e`) represents the co-occurrence frequency of entity e and entity e`; and Sentiment(e,e`) represents the text sentiment intensity between entity e and entity e`, so as to realize real-time mapping of the knowledge graph to the business dynamic changes.

[0059] The model training and optimization module is configured to take a large-scale pre-trained language model as the core, receive the dynamic knowledge graph output by the multi-modal data processing module as the basic data, obtain external knowledge from the basic data by combining a retrieval and enhancement generation technology, perform parameter lightweight fine-tuning by using a low-rank matrix decomposition technology, and optimize the training process by using an adaptive learning rate and a multi-task loss function, so as to generate an optimized model adapted to the business scenario.

[0060] In this embodiment, the model training and optimization module specifically includes:

[0061] The core model adaptation unit is configured to take a large-scale pre-trained language model as the basic engine, obtain external knowledge from the dynamic knowledge graph (containing entity relation information such as industry knowledge and market data) output by the multi-modal data processing module by relying on a retrieval and enhancement generation technology, filter redundant information by using a semantic matching algorithm, generate context input fused with multi-source knowledge, enhance the understanding ability of the model to complex business problems, and perform model lightweight fine-tuning by using a low-rank matrix decomposition (LoRA) technology, decompose a high-dimensional parameter matrix into a low-rank sub-matrix, optimize the key parameters of the model, and reduce the cost of domain adaptation based on private business data.

[0062] The model intelligent training unit is configured to dynamically adjust a learning rate according to a complexity of the output data of the multi-modal data processing module (such as a text entropy value, a table dimension, and an image feature density), to realize stable training in a complex data scenario through an adaptive learning rate formula, and to improve model convergence efficiency. The model intelligent training unit is also configured to jointly optimize an intent classification, SQL query generation, and decision explainability task, to adopt a multi-task loss function, and to ensure that the model optimizes logical transparency while generating a decision result.

[0063] The adaptive learning rate formula is as follows:

[0064]

[0065] In the formula, η t represents an adaptive learning rate at time t, that is, a learning rate adopted by the model at the t-th step of training; η0 represents an initial learning rate, which is a basic learning rate set by the model at the beginning of training, that is, t = 0; λ represents an adjustment coefficient, which is used to control the influence degree of data complexity on the degree of learning rate decay, and can be adjusted according to actual requirements or experimental results; and DataComplexity(t) represents data complexity at time t, which is used to quantify the complexity of the current training data, and the value changes with the change of the data in the training process.

[0066] The calculation formula of the multi-task loss function is as follows:

[0067] L = α1*L intent + α2*L sql + α3*L explain ,

[0068] In the formula, L represents the total loss of the multi-task, which is a weighted combination of the losses of the intent classification, SQL query generation, and decision explainability tasks, and is used to guide parameter updating of the model in the multi-task joint training process; L intent represents the loss of the intent classification task, which is used to measure the difference between the prediction result and the true result of the model in the intent classification task; L sql represents the loss of the SQL query generation task, which is used to evaluate the accuracy of the SQL query generated by the model, and reflects the gap between the performance of the model in the task and the expectation; and L explain represents the loss of the decision explainability task, which is used to reflect the quality of the decision explanation generated by the model, and is calculated based on a local explainable model-independent explanation technology, and is used to optimize the logical transparency of the model decision; α1, α2, and α3 respectively represent weight coefficients of the losses of the intent classification, SQL query generation, and decision explainability tasks, which are used to adjust the importance of different tasks in the total loss, and can be set according to the requirements of each task or experimental results.

[0069] Large-Scale Pre-trained Language Model (LPLM) refers to a large language model that is pre-trained based on massive text data through unsupervised or semi-supervised learning. Its core features are large number of parameters (usually from tens of billions to hundreds of billions), massive training data (covering books, web pages, papers, and other multi-domain text), and learning of human language grammar rules, semantic association, knowledge logic, and context understanding ability through deep neural network architectures such as Transformer.

[0070] The intelligent decision generation module is used to generate preliminary decision suggestions and verify logical rationality, and then evaluate logical bias in the suggestions through feature importance analysis, and finally correct the suggestions by fusing real-time data and domain rules to generate an explainable report containing reasoning process and visual basis.

[0071] In this embodiment, the intelligent decision generation module specifically includes:

[0072] The suggestion generation and verification unit is used to generate preliminary decision suggestions for business problems based on the optimization model output by the model training and optimization module and the dynamic knowledge graph (containing information such as "product-market-consumer" entity relationships) constructed by the multi-modal data processing module, and simultaneously call the business rule engine to verify the rationality of the preliminary suggestions against the basic business logic (for example, for the problem of "Southeast Asia mobile phone sales decline", generate a preliminary direction of "adjusting promotion strategy", and associate the price-sales relationship chain in the knowledge graph to assist verification).

[0073] The suggestion evaluation unit is used to identify logical bias in the preliminary suggestions based on real-time business data (including regional, time, and other dimensional features) provided by the multi-modal data processing module using feature importance analysis techniques, detect whether there are analysis blind spots in the regional and time dimensions by quantifying the contribution of each input feature to the decision result, and investigate possible logical flaws in the preliminary suggestions.

[0074] The correction and report generation unit is used to correct the preliminary suggestions after bias evaluation by combining real-time market data (such as competitor dynamics, user feedback) provided by the multi-modal data processing module and pre-set domain rules (such as compliance constraints); simultaneously generate an explainable report, present the decision logic tree through a visualization tool, and label the reasoning path corresponding to the knowledge graph of the multi-modal data processing module (such as "consumer age layer → purchasing power → promotion response"), thereby improving the transparency and credibility of the decision.

[0075] Wherein, the suggestion evaluation unit specifically adopts a Shapley value algorithm to identify logical bias in the preliminary suggestion, and the formula is as follows:

[0076]

[0077] In the formula, Φ i represents the Shapley value of the i-th feature, which is used to quantify the average marginal contribution of feature i to the decision result; N represents the set of all features, and S represents a subset selected from the set N; represents that the subset S is selected from "all features except the i-th feature"; |S| represents the number of features contained in the subset S; n represents the total number of features, that is, the size of the set N; v(S) represents the decision value of the model when using the subset S; and v(S∪{i}) represents the corresponding decision value after feature i is added to the subset S.

[0078] The dynamic feedback module is configured to take the explainability report output by the intelligent decision generation module as a core input, combine real-time business data provided by the multi-modal data processing module, and build a multi-dimensional evaluation system to quantify the model performance; meanwhile, the dynamic feedback module is configured to collect user feedback to form incremental samples, and through a reinforcement learning algorithm based on a strategy gradient (a reward function integrating evaluation indexes is generated to generate an optimization signal), the dynamic feedback module is configured to act on the model training and optimization module in a reverse manner to realize a closed-loop iteration of "data-decision-feedback-optimization" and link all the process modules for continuous optimization.

[0079] In this embodiment, the dynamic feedback module specifically includes:

[0080] The evaluation system construction unit is configured to take the explainability report output by the intelligent decision generation module as a core input, combine real-time business data (such as market dynamics, user behavior, and entity relationship data) provided by the multi-modal data processing module, and build a multi-dimensional evaluation system covering accuracy, coverage, diversity, and explainability; then, the evaluation system construction unit is configured to calculate the values of each index in real time through a statistical learning method to provide a quantitative basis for model performance evaluation; wherein, the accuracy measures the matching degree of the decision result and the actual business data output by the multi-modal data processing module, the coverage reflects the comprehensiveness of the model in calling multi-modal data dimensions, the diversity reflects the differentiation level of the suggestion scheme, and the explainability quantifies the logical transparency of the decision logic through the logical transparency index in the explainability report.

[0081] The feedback data collection unit is configured to collect feedback data of the user on the output result of the intelligent decision generation module, including business index changes (such as sales, user conversion rate, etc.) monitored by the multi-modal data processing module after the strategy is implemented, and correction records of the artificial on the explainability report and the decision suggestion, and process the collected feedback data into incremental training samples to provide data support for subsequent model optimization;

[0082] The reinforcement learning optimization unit is used to adopt a policy gradient-based reinforcement learning algorithm to link the model training and optimization module for full-process optimization: firstly, the indexes of the multi-dimensional evaluation system are integrated into the reward function, and the pros and cons of the output result of the intelligent decision-making generation module are quantified through the reward function; secondly, the incremental training sample is input, and the real-time data updating learning process of the multi-modal data processing module is combined to guide the model training and optimization module to adjust the pre-training model parameters and optimize the training strategy; finally, the model decision-making ability is continuously updated to form a closed-loop iterative process of "data-decision-feedback-optimization", so as to ensure that the full-link module adapts to the business demand iteration.

[0083] The expression of the reward function is:

[0084]

[0085] In the formula, R total represents the cumulative total reward, which is the total reward obtained by the model in the entire time range from time 0 to time T, and comprehensively reflects the performance in various aspects; t represents the time step, which is used to identify different times of reward calculation, and is sequentially advanced from the initial time 0 to the final time T; T is the total time step, that is, the length of time during which the entire decision or interaction process lasts, and ends at time T; γ is a discount factor, and is valued between 0 and 1, which is used to measure the importance of future rewards at the current time; is a weight coefficient, which is used to adjust the contribution degree of R acc , R exp , R div in the total reward; R acc represents the accuracy reward, which is used to measure the matching degree of the model decision result and the actual data; R exp represents the coverage rate reward, which is used to reflect the comprehensiveness of the data dimensions involved in the model; and R div represents the diversity reward, which is used to reflect the differentiation degree of the suggestion scheme generated by the model.

[0086] As can be seen from the above, the intelligent business decision-making support system based on multi-modal data and knowledge graph can effectively improve the comprehensiveness, accuracy and transparency of business decision-making, and solve the problems of weak heterogeneous data integration capability, poor model adaptability and low decision transparency in existing business analysis.

[0087] The principles and implementation modes of the application are described in detail by using the above application specific examples, and these examples are only used to help understand the core technical content of the application. Based on the above specific embodiments of the application, any improvement and modification of the application made by those skilled in the art without departing from the principles of the application shall fall within the patent protection scope of the application.

Claims

1. An intelligent business decision support system based on multi-modal data and knowledge graph, characterized in that, It comprises: A multi-modal data processing module for collecting text, table, and image multi-modal data, and constructing a knowledge graph based on an entity-relation model after differential preprocessing and feature extraction, realizing real-time semantic mapping of business dynamics; A model training and optimization module for receiving dynamic knowledge graphs output by the multi-modal data processing module as basic data, combining retrieval enhancement generation technology to obtain external knowledge, and using low-rank matrix decomposition technology for parameter lightweight fine-tuning, and through adaptive learning rate and multi-task loss function optimization training process, generating an optimized model adapted to the business scenario; An intelligent decision generation module for generating preliminary decision suggestions based on the optimized model output by the model training and optimization module, and verifying the logical rationality, then evaluating the logical bias in the suggestions through feature importance analysis, and finally correcting the suggestions by fusing real-time data and domain rules to generate an explainability report containing reasoning process and visual basis; A dynamic feedback module for constructing a multi-dimensional evaluation system to quantify model performance based on the explainability report output by the intelligent decision generation module and real-time business data from the multi-modal data processing module; at the same time, collecting user feedback to form incremental samples, and through the reinforcement learning algorithm based on policy gradient, the model training and optimization module is reversely affected to realize the closed-loop iteration of "data-decision-feedback-optimization". 2.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 1, wherein, The multi-modal data processing module specifically comprises: A multi-modal data acquisition unit for acquiring text, table, and image multi-modal data; A differential preprocessing unit for starting differential preprocessing procedures according to the characteristics of different types of data: performing semantic analysis preprocessing on text data to lay a foundation for sentiment analysis and entity recognition; converting the structure of table data to adapt to the low-dimensional vector conversion requirement; extracting visual key information from image data through target detection technology to prepare for structured conversion, ensuring that all types of data are converted into effective formats for feature analysis; A multi-modal feature extraction unit for deep feature extraction of preprocessed multi-modal data, specifically including: for text data, using a deep learning model based on attention mechanism to carry out sentiment analysis and named entity recognition, extracting semantic features of user behavior keywords and sentiment orientation; for table data, using a visual Transformer transfer learning architecture to capture the correlation between rows and columns through a hierarchical attention mechanism, converting table data into low-dimensional embedding vectors, supporting multi-dimensional feature aggregation of time, geography, and product category, realizing dynamic aggregation analysis of sales data by quarter and region; for image data, using a convolutional neural network combined with YOLO series target detection technology to identify key information such as price labels and promotion signs in product images, and then converting visual information into structured data through OCR technology to supplement entity details for the knowledge graph; The knowledge graph construction unit is configured to extract three types of entities, i.e., "product", "market" and "consumer", and the associated relationships of "sales-geographical area-growth" and "user-preference-product" from the multi-modal features based on an entity-relation model, and construct a knowledge graph. The knowledge graph updating unit is configured to learn semantic associations between entities by using a graph neural network algorithm, introduce time dimension weights, and dynamically adjust edge weights of the graph according to data update frequencies, so that the knowledge graph can map dynamic changes in business in real time. 3.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 2, wherein, The knowledge graph updating unit dynamically adjusts the edge weights of the graph by the following formula, so as to realize real-time mapping of the knowledge graph to business dynamics: ω e→e` = σ1 * Cooccurrence(e, e') + σ2 * Sentiment(e, e'), In the formula, ω e→e represents the weight of the edge between entity e and entity e` in the knowledge graph; σ1 and σ2 are weight coefficients for adjusting the influence degree of the co-occurrence frequency of entities and the text sentiment intensity on the edge weight; and σ1+σ2=1; Cooccurrence(e, e`) represents the co-occurrence frequency of entity e and entity e`, and Sentiment(e, e`) represents the text sentiment intensity between entity e and entity e`, realizing real-time mapping of the knowledge graph to business dynamics. 4.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 2, wherein, The model training and optimization module specifically includes: The core model adaptation unit is configured to take a large-scale pre-trained language model as a basic engine, rely on a retrieval enhancement generation technology to retrieve dynamic knowledge graphs output by the multi-modal data processing module to obtain external knowledge, filter redundant information by a semantic matching algorithm, generate context input fused with multi-source knowledge, and enhance the understanding ability of the model to complex business problems; and introduce a low-rank matrix decomposition technology to perform model light-weight fine-tuning, decompose a high-dimensional parameter matrix into a low-rank sub-matrix, and optimize key parameters of the model. The model intelligent training unit is configured to dynamically adjust a learning rate according to the complexity of the data output by the multi-modal data processing module, realize stable training in a complex data scenario by using an adaptive learning rate formula, and improve the convergence efficiency of the model; and jointly optimize intent classification, SQL query generation and decision interpretability tasks, use a multi-task loss function, and ensure that the model optimizes logical transparency while generating decision results. 5.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 4, wherein, The adaptive learning rate formula is as follows: In the formula, η t denotes the adaptive learning rate at time t, i.e. the learning rate adopted by the model when training to the t-th step; η0 represents an initial learning rate, which is a basic learning rate set by the model at the beginning of training, i.e., t=0; λ represents an adjustment coefficient, which is used to control the influence of data complexity on the degree of learning rate decay, and can be adjusted according to actual needs or experimental results; DataComplexity(t) represents the data complexity at time t, which is used to quantify the complexity of the current training data, and its value will change with the change of the data during the training process. 6.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 5, wherein, The calculation formula of the multi-task loss function is as follows: L = a1*L intent + a2*L sql + a3*L explain , In the formula, L represents the total loss of multi-task, which is the overall loss of the three task losses of intent classification, SQL query generation and decision explainability after weighted combination, used to guide the parameter update of the model in the multi-task joint training process; L intent represents the loss of the intent classification task, used to measure the difference between the prediction result and the true result of the model on the intent classification task; L sql represents the loss of the SQL query generation task, used to evaluate the accuracy of the SQL query generated by the model, and reflects the gap between the performance of the model on the task and the expectation; L explain represents the loss of the decision explainability task, used to reflect the quality of the decision explanation generated by the model, calculated based on the local explainable model independent explanation technology, and used to optimize the logical transparency of the model decision; α1, α2, α3 represent the weight coefficients of the task losses of intent classification, SQL query generation and decision explainability respectively, used to adjust the importance of different tasks in the total loss, which can be set according to the requirements of each task or the experimental results. 7.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 4, wherein, The intelligent decision generation module specifically includes: The suggestion generation and verification unit is configured to generate preliminary decision suggestions for business problems in combination with the dynamic knowledge graph constructed by the multi-modal data processing module, relying on the optimized model output by the model training and optimization module, and simultaneously call a business rule engine to verify the rationality of the preliminary suggestions against basic business logic; The suggestion evaluation unit is configured to identify logical biases in the preliminary suggestions by using feature importance analysis technology based on real-time business data provided by the multi-modal data processing module, detect whether there are analysis blind spots in the regional dimension and the time dimension by quantifying the contribution of each input feature to the decision result, and troubleshoot possible logical vulnerabilities in the preliminary suggestions. The correction and report generation unit is configured to correct the preliminary suggestion based on the real-time market data provided by the multi-modal data processing module and the preset field rules, and generate an explainability report, present a decision logic tree through a visualization tool, and mark an inference path corresponding to the knowledge graph of the multi-modal data processing module, so as to improve the transparency and credibility of the decision. 8.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 7, wherein, The suggestion evaluation unit specifically adopts a Shapley value algorithm to identify logical bias in the preliminary suggestion, and the formula is as follows: where Φ i represents the Shapley value of the i-th feature, which quantifies the average marginal contribution of feature i to the decision outcome; N represents the set of all features, and S represents a subset selected from the set N; represents that the subset S is selected from "all features except the i-th feature"; |S| represents the number of features contained in the subset S; n represents the total number of features, i.e., the size of the set N; v(S) represents the decision value of the model when using the subset S; and v(S∪{i}) represents the corresponding decision value after adding the feature i to the subset S. 9.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 7, wherein, The dynamic feedback module specifically includes: The evaluation system construction unit is configured to take the explainability report output by the intelligent decision generation module as a core input, combine real-time business data provided by the multi-modal data processing module, and construct a multi-dimensional evaluation system covering accuracy, coverage, diversity and explainability; subsequently, the evaluation system construction unit is configured to calculate the values of the indexes in real time through a statistical learning method, and provide a quantitative basis for model performance evaluation; wherein, the accuracy measures the matching degree between the decision result and the actual business data output by the multi-modal data processing module, the coverage reflects the comprehensiveness of the model in calling multi-modal data dimensions, the diversity reflects the differentiation level of the suggestion scheme, and the explainability quantifies the logical transparency through the logical transparency index in the explainability report to evaluate the clarity of the decision logic; The feedback data acquisition unit is configured to acquire feedback data of the user on the output result of the intelligent decision generation module, including a change in a business index monitored by the multi-modal data processing module after implementation of the strategy, a correction record of the explainability report and the decision suggestion by the artificial, and process the acquired feedback data into an incremental training sample to provide data support for subsequent model optimization; The reinforcement learning optimization unit is configured to adopt a reinforcement learning algorithm based on a policy gradient, and link the model training and optimization module to perform full-process optimization: firstly, the reinforcement learning optimization unit is configured to integrate the indexes of the multi-dimensional evaluation system into a reward function, and quantify the pros and cons of the output result of the intelligent decision generation module through the reward function; secondly, the reinforcement learning optimization unit is configured to take the incremental training sample as an input, combine a real-time data update learning process of the multi-modal data processing module, guide the model training and optimization module to adjust pre-training model parameters and optimize training strategies; and finally, the reinforcement learning optimization unit is configured to continuously update the decision-making ability of the model, form a closed-loop iterative process of "data-decision-feedback-optimization", and ensure that the full-link module adapts to the iterative business requirements. 10.The intelligent business decision support system based on multi-modal data and knowledge graph of claim 7, wherein, The expression of the reward function is as follows: In the formula, R total represents the cumulative total reward, which is the total reward obtained by the model in the entire time range from time 0 to time T, integrating the performance of all aspects; t represents a time step, used to identify different time instants for reward calculation, sequentially advancing from the initial time instant 0 to the final time instant T; T is the total number of time steps, that is, the length of time during which the entire decision or interaction process lasts, ending at the time instant T; γ is a discount factor, taking a value between 0 and 1, used to measure the importance of future rewards in the present; are weight coefficients, respectively used to adjust the contribution of R acc , R exp , R div to the total reward; R acc represents an accuracy reward, used to measure the matching degree of the model decision result and the actual data; R exp represents a representative coverage reward, used to reflect the comprehensiveness of the data dimensions involved in the model; R div represents a diversity reward, used to reflect the differentiation degree of the model generated suggestion scheme.

Images