Decision-making method and device based on multi-modal data and electronic equipment
By constructing a comprehensive profile using multimodal data and extracting target features using policy optimization models and deep learning methods, the accuracy and interpretability issues of single-modal data decision-making methods are solved, enabling efficient decision support in the financial and industrial sectors.
Patent Information
- Application Number
- CN202511526229.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-23
- Publication Date
- 2026-01-30
AI Technical Summary
Existing decision-making methods based on single-modal data have poor accuracy and interpretability in the financial and industrial sectors, making it difficult to meet the analysis needs of complex data.
A comprehensive profile is constructed using multimodal data. Target features are extracted through policy optimization models and deep learning methods. The SHAP algorithm is then used to conduct causal effect analysis, generate decision-making schemes, and improve the accuracy and interpretability of decision-making.
It enables automated risk assessment and real-time early warning based on multimodal data, improving the accuracy and interpretability of decision-making, ensuring data security, and supporting cross-platform model deployment.
Smart Images
Figure CN121436713A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence, and in particular to a decision-making method and device based on multi-modal data and an electronic device. BACKGROUND
[0002] With the development of artificial intelligence, in the intelligent decision-making system in the fields of finance, industry, etc., how to improve the accuracy and explainability of decision-making is a key problem to be solved. The traditional decision-making method mainly relies on single modal data, and usually uses statistical models or simple machine learning models for data analysis. When dealing with complex data, the transparency and explainability of the model are poor, which affects the accuracy and explainability of the decision-making. SUMMARY
[0003] The present application provides a decision-making method and device based on multi-modal data and an electronic device to solve the defects of poor accuracy and explainability of the prior art based on single modal data for decision-making.
[0004] The present application provides a decision-making method based on multi-modal data, comprising: obtaining multi-modal data of a target enterprise; processing the multi-modal data to construct a panoramic portrait of the target enterprise; extracting features from the panoramic portrait of the target enterprise to obtain a plurality of candidate features of the target enterprise, and determining a plurality of target features from the plurality of candidate features; inputting the plurality of target features into a pre-constructed analysis model to obtain a data analysis result of the target enterprise output by the analysis model; the analysis model is used to analyze the plurality of target features based on a pre-set analysis index; generating a decision-making scheme for the target enterprise based on the data analysis result of the target enterprise.
[0005] In some embodiments, the determination of the plurality of target features from the plurality of candidate features comprises: constructing an intervention variable for each candidate feature, and calculating a causal effect value of each candidate feature and the analysis index; determining a plurality of target features from the plurality of candidate features according to the causal effect value of each candidate feature and the data analysis index.
[0006] In some embodiments, the construction of the intervention variable for each candidate feature and the calculation of the causal effect value of each candidate feature and the analysis index comprise: inputting each candidate feature into a pre-constructed strategy optimization model to obtain an intervention variable of each candidate feature output by the strategy optimization model; Based on the intervention variables of each candidate feature, calculate the change in the analytical indicators corresponding to each candidate feature; Based on the intervention variables of each candidate feature and the change in the analytical indicators corresponding to each candidate feature, calculate the causal effect value between each candidate feature and the analytical indicators. The strategy optimization model is trained based on sample states, sample actions, and a reward function. The sample states include multiple sample candidate features, the sample actions include sample intervention variables of multiple sample candidate features, and the reward function is constructed based on the changes in the analysis indicators.
[0007] In some embodiments, the analytical indicators include: access analysis indicators, risk analysis indicators, and credit analysis indicators; the multiple target features include: a first target feature related to the access analysis indicators, a second target feature related to the risk analysis indicators, and a third target feature related to the credit analysis indicators; the analytical model includes: an access analysis model, a risk analysis model, and a credit scoring model; the data analysis results include: access probability, risk score, and credit score.
[0008] In some embodiments, generating a decision-making plan for the target enterprise based on the data analysis results of the target enterprise includes: Determine the weights of the admission probability, the risk score, and the credit score; Based on the weights of the admission probability, risk score, and credit score, as well as the admission probability, risk score, and credit score, a decision-making scheme for the target enterprise is determined.
[0009] In some embodiments, inputting the plurality of target features into a pre-built analysis model to obtain the data analysis results of the target enterprise output by the analysis model includes: The first target feature is input into the admission analysis model to obtain the admission probability of the target enterprise output by the admission analysis model; If the admission probability is greater than or equal to a preset probability threshold, the second target feature is input into the risk analysis model to obtain the risk score of the target enterprise output by the risk analysis model. If the risk score is less than or equal to a preset scoring threshold, the third target feature is input into the credit scoring model to obtain the credit score of the target enterprise output by the credit scoring model.
[0010] In some embodiments, the multimodal data includes: public opinion data related to the target enterprise and credit data of the target enterprise, and the processing of the multimodal data includes: Identify sensitive information in the credit data and perform desensitization processing on the sensitive information; Sentiment recognition is performed on the public opinion data to obtain the sentiment recognition result of the public opinion data.
[0011] In some embodiments, the training process of the analysis model includes: Obtain multimodal data of sample enterprises and determine the data analysis result labels of the sample enterprises; The multimodal data of the samples are processed to construct a comprehensive profile of the sample enterprises. Feature extraction is performed on the panoramic portrait of the sample enterprises to obtain multiple sample candidate features of the sample enterprises, and multiple sample target features are determined from the multiple sample candidate features; Using the target features of the multiple samples as training samples and the data analysis result labels of the sample enterprises as sample labels, an initial analysis model is trained, and after training, the analysis model is obtained.
[0012] The present invention also provides a decision-making device based on multimodal data, comprising: The acquisition unit is used to acquire multimodal data of the target enterprise; The data processing unit is used to process the multimodal data and construct a panoramic profile of the target enterprise. The feature extraction unit is used to extract features from the panoramic portrait of the target enterprise, obtain multiple candidate features of the target enterprise, and determine multiple target features from the multiple candidate features; A data analysis unit is used to input the multiple target features into a pre-built analysis model to obtain the data analysis results of the target enterprise output by the analysis model; the analysis model is used to analyze the multiple target features based on preset analysis indicators; The decision-making unit is used to generate a decision-making plan for the target enterprise based on the data analysis results of the target enterprise.
[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the decision-making method based on multimodal data as described above.
[0014] This invention provides a decision-making method, apparatus, and electronic device based on multimodal data. By integrating multimodal data of the target enterprise, such as credit information and public opinion data, a comprehensive portrait of the target enterprise is constructed. Based on a strategy optimization model and combined with the Shapley Additive Explanations (SHAP) algorithm, features are extracted from the comprehensive portrait of the target enterprise to obtain multiple target features. The strategy optimization model is trained using reinforcement learning. These multiple target features are input into a pre-constructed analysis model to obtain the data analysis results of the target enterprise. The analysis model is trained using deep learning. Based on the data analysis results of the target enterprise, a decision-making scheme for the target enterprise is generated, improving the accuracy and interpretability of the decision-making process and enabling automated risk assessment and real-time early warning. By transforming the analysis model, an Open Neural Network Exchange (ONNX) model is obtained, enabling efficient cross-platform deployment of the model. Dynamic data encryption ensures data security. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0016] Figure 1 This is a flowchart illustrating the decision-making method based on multimodal data provided in an embodiment of the present invention.
[0017] Figure 2 This is a schematic diagram of the process for determining multiple target features from multiple candidate features provided in an embodiment of the present invention.
[0018] Figure 3 This is a schematic diagram of the process for calculating the causal effect value of each candidate feature and analysis index provided in an embodiment of the present invention.
[0019] Figure 4 This is a schematic diagram of the process of inputting multiple target features into a pre-built analysis model, as provided in an embodiment of the present invention.
[0020] Figure 5 This is a flowchart illustrating the process of determining the decision-making scheme for the target enterprise as provided in an embodiment of the present invention.
[0021] Figure 6 This is a flowchart illustrating the training process of the analysis model provided in this embodiment of the invention.
[0022] Figure 7 This is a schematic diagram of the structure of the decision-making device based on multimodal data provided in an embodiment of the present invention.
[0023] Figure 8 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0025] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, without limiting the number of objects; for example, the first object can be one or more.
[0026] Figure 1 This is a flowchart illustrating a decision-making method based on multimodal data provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps: steps 110 to 150. This method flow is merely one possible implementation of the present invention.
[0027] Step 110: Obtain multimodal data of the target company.
[0028] Among them, the multimodal data of the target enterprise refers to a heterogeneous data set obtained from different channels that can reflect the enterprise's status from multiple perspectives.
[0029] Optionally, multimodal data includes: public opinion data related to the target company and credit data of the target company.
[0030] Among them, public opinion data refers to textual information such as public opinions, media reports, and market comments about the target company from public channels. This data is usually unstructured. Credit data refers to structured data used to assess a company's credit status and solvency. This data is usually quantitative and has clearly defined fields.
[0031] Optionally, public opinion data of the target company can be obtained through news media, social media, research reports, and other channels; credit data of the target company can be obtained through business information systems, financial institutions, enterprise platforms, and other channels.
[0032] It should be noted that public opinion data is used to analyze a company's reputational risk, market image, public attention, potential crises, and industry position. For example, a large number of negative reports about litigation may indicate operational difficulties for the company. Credit data is used to quantitatively analyze a company's financial health, solvency, operational stability, and historical credit history. For example, an excessively high debt-to-equity ratio, consecutive negative profits, and a record of defaulting on debts are all high-risk signals.
[0033] Step 120: Process the multimodal data to construct a panoramic profile of the target enterprise.
[0034] The panoramic profile of the target company includes multi-dimensional information about the target company, such as basic information, financial information, risk information, public opinion information, and operational information.
[0035] Optionally, the multimodal data can be preprocessed, including data cleaning, standardization, and data alignment, to obtain preprocessed multimodal data. The preprocessed multimodal data can then be further processed with dynamic encryption and sentiment recognition.
[0036] For example, sensitive information in credit data can be identified and de-identified to ensure data security; sentiment analysis can be performed on public opinion data to obtain the sentiment analysis results.
[0037] Specifically, the system identifies sensitive information in credit data, such as ID card numbers, bank account numbers, and mobile phone numbers; it partially hides or masks sensitive information as needed, displaying only a portion of the information and replacing the remaining characters with specific symbols. It performs word segmentation and sentiment analysis on public opinion data to generate a public opinion sentiment score. The system then links and integrates public opinion data with credit data; it periodically acquires the latest public opinion data to update the overall profile of the target company.
[0038] Optionally, entity extraction is performed on the processed multimodal data to obtain multiple entities, such as enterprises, people, events, and indicators, and the relationships between multiple entities are determined; a knowledge graph of the target enterprise is constructed with entities as nodes and the relationships between entities as edges; and a panoramic profile of the target enterprise is constructed based on the knowledge graph of the target enterprise.
[0039] Optionally, a distributed storage system can be used to store the comprehensive profile of the target enterprise, ensuring high availability and fault tolerance of the data; the data can be backed up regularly, using a combination of incremental and full backups to ensure the integrity and consistency of the backup data.
[0040] Step 130: Extract features from the panoramic profile of the target company to obtain multiple candidate features of the target company, and determine multiple target features from the multiple candidate features.
[0041] Among them, multiple candidate features refer to the set of all original features that may contribute to the final decision analysis, which are initially extracted from the panoramic profile of the target company; multiple candidate features include at least candidate features such as debt-to-equity ratio, profit margin, and cash flow.
[0042] Among them, multiple target features refer to the subset of features that are highly relevant to a specific analysis task and have high predictive value, which are finally determined from multiple candidate features after a series of rigorous statistical and screening processes.
[0043] Optionally, the decision-making task and analysis indicators are determined, and multiple target features are determined from multiple candidate features based on the decision-making task and analysis indicators.
[0044] It's important to note that target characteristics are typically business-understandable metrics, rather than difficult-to-interpret intermediate parameters such as debt-to-equity ratio or the number of administrative penalties in the past six months. This makes data analysis results based on target characteristics easier for business experts to understand and trust, thus meeting compliance requirements in fields such as finance and risk control.
[0045] Step 140: Input multiple target features into the pre-built analysis model to obtain the data analysis results of the target enterprise output by the analysis model; the analysis model is used to analyze multiple target features based on preset analysis indicators.
[0046] The analysis model includes one or more neural network models; the analysis metrics are predefined quantitative standards used to measure and express the results of data analysis.
[0047] Optionally, the data analysis results may include information such as ratings, levels, probabilities, category labels, and key criteria.
[0048] Optionally, preset analytical indicators can be determined based on decision-making business needs or user input.
[0049] For example, in credit risk control scenarios, the analytical indicators can be credit scores, default probabilities, or risk levels.
[0050] Step 150: Based on the data analysis results of the target company, generate a decision-making plan for the target company.
[0051] In this context, a decision-making plan refers to specific, actionable recommendations or instructions, such as rejecting a loan application. The decision-making plan is the final output of the entire data processing workflow, aiming to directly translate the insights gained from data analysis into business action.
[0052] Optionally, based on the data analysis results and combined with preset business rules and strategies, a preliminary decision-making plan is generated; the preliminary decision-making plan is then optimized to obtain the decision-making plan for the target enterprise.
[0053] In this embodiment of the invention, multimodal data of the target enterprise is acquired; the multimodal data is processed to construct a panoramic profile of the target enterprise; features are extracted from the panoramic profile of the target enterprise to obtain multiple candidate features of the target enterprise; multiple target features are determined from the multiple candidate features; the multiple target features are input into a pre-built analysis model to obtain the data analysis results of the target enterprise output by the analysis model; and a decision-making scheme for the target enterprise is generated based on the data analysis results of the target enterprise, thereby improving the accuracy and interpretability of the decision-making.
[0054] Figure 2 This is a schematic diagram illustrating the process of determining multiple target features from multiple candidate features, provided in an embodiment of the present invention. Figure 2 As shown, in some embodiments, step 130 involves determining multiple target features from multiple candidate features, including: Step 131: Construct intervention variables for each candidate feature and calculate the causal effect value between each candidate feature and the analysis index; Among them, the causal effect value is used to quantify the impact of a change in one variable on a change in another variable.
[0055] Optionally, confounding factors, i.e. variables that simultaneously affect candidate features and analytical indicators, can be identified, and a causal inference mathematical model can be used to estimate the impact of intervention variables on the analysis results while controlling for confounding factors.
[0056] Step 132: Determine multiple target features from multiple candidate features based on the causal effect value of each candidate feature and the data analysis index.
[0057] Optionally, multiple target features can be determined from multiple candidate features based on the causal effect value of each candidate feature and the data analysis indicator, as well as a preset causal effect threshold.
[0058] In this embodiment of the invention, by constructing intervention variables for each candidate feature and calculating the causal effect value between each candidate feature and the analysis index, multiple target features are determined from multiple candidate features, thereby improving the interpretability of the data analysis results output by the analysis model.
[0059] Figure 3 This is a flowchart illustrating the calculation of the causal effect value between each candidate feature and the analysis index, provided for embodiments of the present invention. In some embodiments, step 131, which involves constructing the intervention variable for each candidate feature and calculating the causal effect value between each candidate feature and the analysis index, includes: Step 1311: Input each candidate feature into the pre-built policy optimization model to obtain the intervention variable of each candidate feature output by the policy optimization model; Optionally, the policy optimization model is a reinforcement learning model, such as the Proximal Policy Optimization (PPO) model.
[0060] The policy optimization model includes a policy network and a value network; the value network is used to calculate the advantage function, which helps the policy network to be updated better.
[0061] The strategy optimization model is trained based on sample states, sample actions, and reward functions. Sample states include multiple sample candidate features, sample actions include sample intervention variables of multiple sample candidate features, and the reward function is constructed based on the changes in the analysis indicators.
[0062] Step 1312: Based on the intervention variables of each candidate feature, calculate the change in the analytical indicators corresponding to each candidate feature; Optionally, the causal effect value can be combined with the SHAP value to assess the contribution and causal impact of candidate features on the prediction results of the analytical model.
[0063] Step 1313: Calculate the causal effect value between each candidate feature and the analysis indicator based on the intervention variable of each candidate feature and the change in the corresponding analysis indicator. Optionally, by comparing counterfactual results with actual results, the causal effect value of each candidate feature is calculated, and features that have a significant causal impact on the change of the analysis index are selected based on the magnitude of the causal effect value.
[0064] Optionally, the training process of the policy optimization model includes: Build status (Including candidate features such as debt-to-equity ratio, profit margin, and cash flow), define the action. (Intervention on features), defining rewards (Analyze changes in indicators); Divide the dataset into training and testing sets. Define the interaction logic of the environment: given a state. Execute actions Return to the new state and rewards ; Initialize policy network and value function ; Interact with the environment using the current strategy to collect a batch of trajectory data, including information such as status, actions, and rewards; The advantage function at each time step is calculated using the generalized advantage estimation algorithm. Update policy network parameters using the shearing objective function. The value network parameters are updated using the mean squared error (MSE) loss function. Repeat the above steps until the policy network parameters converge.
[0065] In some embodiments, the analytical indicators include: access analysis indicators, risk analysis indicators, and credit analysis indicators; multiple target features include: a first target feature related to the access analysis indicators, a second target feature related to the risk analysis indicators, and a third target feature related to the credit analysis indicators; the analytical models include: an access analysis model, a risk analysis model, and a credit scoring model; and the data analysis results include: access probability, risk score, and credit score.
[0066] The first target characteristic refers to the characteristics strongly related to market access. For example, whether the registered capital meets the requirements, whether the company is on the list of dishonest persons subject to enforcement, and whether the main business is compliant; these are usually hard indicators.
[0067] The second target characteristic refers to features strongly correlated with risk. Examples include: cash flow volatility, the number of negative public opinions, legal litigation history, and debt-to-equity ratio.
[0068] The third target characteristic refers to features strongly correlated with credit. Examples include: historical performance record, profitability, quality of upstream and downstream partners, and industry standing.
[0069] The admission analysis model can be a binary classification model, such as logistic regression or XGBoost; it takes a first target feature as input and outputs an admission probability. The risk analysis model can be a regression model or a multi-class classification model; it takes a second target feature as input and outputs a risk score. The credit scoring model can be a regression model; it takes a third target feature as input and outputs a credit score. The three models can run in parallel, processing their respective feature sets.
[0070] Figure 4 This is a schematic diagram illustrating the process of inputting multiple target features into a pre-built analysis model, as provided in an embodiment of the present invention. Figure 4 As shown, in some embodiments, step 140 inputs multiple target features into a pre-built analysis model to obtain the data analysis results of the target enterprise output by the analysis model, including: Step 141: Input the first target feature into the admission analysis model to obtain the admission probability of the target enterprise output by the admission analysis model; Step 142: If the admission probability is greater than or equal to the preset probability threshold, input the second target feature into the risk analysis model to obtain the risk score of the target enterprise output by the risk analysis model. Optionally, it can be determined whether the admission probability of the target enterprise is greater than or equal to a preset probability threshold. If so, it means that the target enterprise has passed the initial screening. The second target feature is then input into the risk analysis model to obtain the risk score of the target enterprise output by the risk analysis model. Otherwise, it means that the target enterprise has not met the minimum admission standard, and the process is terminated immediately.
[0071] Step 143: If the risk score is less than or equal to the preset scoring threshold, input the third target feature into the credit scoring model to obtain the credit score of the target enterprise output by the credit scoring model.
[0072] Optionally, it can be determined whether the risk score of the target enterprise is less than or equal to a preset scoring threshold. If so, it indicates that the risk of the target enterprise is controllable, and the third target feature is input into the credit scoring model to obtain the credit score of the target enterprise output by the credit scoring model; otherwise, it indicates that the risk of the target enterprise is too high, and the process is terminated.
[0073] In this embodiment of the invention, by sequentially employing an admission analysis model, a risk analysis model, and a credit scoring model to analyze different target features, the efficiency of data analysis is improved, the computational cost is reduced, and the interpretability of the data analysis results is enhanced.
[0074] Figure 5 This is a flowchart illustrating the process of determining the decision-making scheme for a target enterprise, as provided in an embodiment of the present invention. Figure 5 As shown, in some embodiments, step 150 generates a decision-making plan for the target enterprise based on the data analysis results of the target enterprise, including: Step 151: Determine the weights of the admission probability, risk score, and credit score; Optionally, the weights of the admission probability, risk score, and credit score can be dynamically updated.
[0075] Step 152: Based on the weights of the admission probability, risk score, and credit score, as well as the admission probability, risk score, and credit score, determine the decision-making scheme for the target company.
[0076] Optionally, the first target feature is input into the admission analysis model to obtain the admission probability of the target enterprise output by the admission analysis model; the second target feature is input into the risk analysis model to obtain the risk score of the target enterprise output by the risk analysis model; and the third target feature is input into the credit scoring model to obtain the credit score of the target enterprise output by the credit scoring model.
[0077] In this embodiment of the invention, the decision-making scheme for the target enterprise is determined by using the weights of the admission probability, the risk score, and the credit score, as well as the admission probability, risk score, and credit score, thereby improving the accuracy of the decision-making.
[0078] Figure 6 This is a flowchart illustrating the training process of the analysis model provided in an embodiment of the present invention. Figure 6 As shown, in some embodiments, the training process of the analysis model includes: Step 610: Obtain multimodal data of the sample enterprises and determine the data analysis result labels of the sample enterprises; Optionally, the multimodal data of the sample includes at least sample public opinion data and sample credit data of the sample enterprises.
[0079] Step 620: Process the multimodal data of the samples to construct a comprehensive profile of the sample enterprises; Optionally, sensitive information in the sample credit data is identified and desensitized; sentiment recognition is performed on the sample public opinion data to obtain the sentiment recognition results of the sample public opinion data.
[0080] Step 630: Extract features from the overall profile of the sample enterprises to obtain multiple candidate features of the sample enterprises, and determine multiple target features of the samples from the multiple candidate features. Optionally, feature extraction is performed on the overall profile of the sample enterprises based on the strategy optimization model to obtain multiple sample target features. These multiple sample target features include at least: a first sample target feature related to access analysis indicators, a second sample target feature related to risk analysis indicators, and a third sample target feature related to credit analysis indicators.
[0081] Step 640: Using multiple target features of the samples as training samples and the data analysis result labels of the sample enterprises as sample labels, train the initial analysis model. After training, the analysis model is obtained.
[0082] Optionally, the analysis indicators include: access analysis indicators, risk analysis indicators, and credit analysis indicators; the initial analysis models include: initial access analysis model, initial risk analysis model, and initial credit scoring model.
[0083] Optionally, DeepSeek can be guided to generate admission rules using deep learning and attention mechanisms.
[0084] Optionally, the initial admission analysis model and the initial credit scoring model can be trained using the cross-entropy loss function.
[0085] Optionally, DeepSeek can be guided to use isolated trees to build an initial risk analysis model, such as an anti-fraud rule model.
[0086] Optionally, DeepSeek can be guided to use stacking methods combined with XGBoost, random forest, and logistic regression to train the initial credit scoring model.
[0087] Optionally, the trained admission analysis model, risk analysis model, and credit scoring model can be converted into ONNX models. ONNX is an open, neutral model representation format standard whose core goal is to solve interoperability issues in the artificial intelligence ecosystem.
[0088] Optionally, operations such as quantization, graph optimization, and model compression can be performed on the ONNX model to improve its inference performance and efficiency on various hardware.
[0089] It should be noted that the ONNX model can be deployed on a wide variety of hardware and environments, including cloud servers, edge computing devices, mobile applications, etc.
[0090] Optionally, the specific steps for graph optimization of the ONNX model are as follows: Load the ONNX model, parse the model's graph structure, and extract nodes, input / output tensors, and initializers; Use the official ONNX optimization tools to perform predefined optimization processes, such as eliminating redundant nodes and fusing convolutional layers and activation functions; Check whether the optimized model maintains functional consistency, such as unchanged input / output dimensions and prediction accuracy.
[0091] Optionally, the specific steps for compressing the ONNX model are as follows: Use L1 / L2 regularization to identify redundant weights and set them to zero proportionally. Based on channel importance scoring, such as the L1 norm of channel output, redundant channels are deleted. Accelerate inference using sparse computing libraries; Use model simplification tools to merge redundant operators; Performance verification was performed using the same independent test set as the original model, and the data analysis results of the model before and after compression were compared. A performance degradation threshold is set. If the performance degradation value of the compressed model exceeds the performance degradation threshold, the pruning ratio is automatically adjusted through the Bayesian optimization algorithm to balance the model compression rate and performance degradation.
[0092] Optionally, model inference and result feedback can be performed through the ONNX Runtime, with the following specific steps: Load the optimized ONNX model using the ONNX Runtime API; Enable thread pool optimization, GPU acceleration, and dynamic batch processing. Assign a unique identifier to each model and ensure consistency between the model and business rules by matching the version number; Receive multimodal data from the decision engine, such as numerical features, text embeddings, graph embeddings, etc., and perform tensor transformation on the multimodal data according to the model input requirements; The input data dimensions are validated. If abnormal data is found, an alarm is triggered and the data is returned to the data source for reprocessing. Call the session.run() method to perform inference, record the inference time, and ensure that the response time is lower than the preset time threshold; Output data analysis results; Return the structured reasoning results to the decision engine; Record complete inference pipeline log data, such as input data hash, model version, inference results, and time consumption, and store it for auditing and traceability.
[0093] The decision-making device based on multimodal data provided in the embodiments of the present invention will be described below. The decision-making device based on multimodal data described below and the decision-making method based on multimodal data described above can be referred to and correspond to each other.
[0094] Figure 7 This is a schematic diagram of the structure of a decision-making device based on multimodal data provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the decision-making device 700 based on multimodal data includes: Acquisition unit 710 is used to acquire multimodal data of the target enterprise; The data processing unit 720 is used to process multimodal data and construct a panoramic profile of the target enterprise; The feature extraction unit 730 is used to extract features from the panoramic portrait of the target enterprise, obtain multiple candidate features of the target enterprise, and determine multiple target features from the multiple candidate features; The data analysis unit 740 is used to input multiple target features into a pre-built analysis model to obtain the data analysis results of the target enterprise output by the analysis model; the analysis model is used to analyze multiple target features based on preset analysis indicators; Decision unit 750 is used to generate decision-making solutions for the target company based on the data analysis results of the target company.
[0095] Optionally, multiple target features are determined from multiple candidate features, including: Construct intervention variables for each candidate feature and calculate the causal effect value between each candidate feature and the analysis index; Based on the causal effect value of each candidate feature and the data analysis indicator, multiple target features are determined from multiple candidate features.
[0096] Optionally, an intervention variable is constructed for each candidate feature, and the causal effect value between each candidate feature and the analysis index is calculated, including: Each candidate feature is input into a pre-built policy optimization model to obtain the intervention variable for each candidate feature output by the policy optimization model. Based on the intervention variables for each candidate feature, the change in the corresponding analytical indicators for each candidate feature is calculated. Based on the intervention variables of each candidate feature and the change in the corresponding analytical indicators, calculate the causal effect value between each candidate feature and the analytical indicators. The strategy optimization model is trained based on sample states, sample actions, and reward functions. Sample states include multiple sample candidate features, sample actions include sample intervention variables of multiple sample candidate features, and the reward function is constructed based on the changes in the analysis indicators.
[0097] Optionally, the analysis indicators include: access analysis indicators, risk analysis indicators, and credit analysis indicators; multiple target features include: a first target feature related to the access analysis indicators, a second target feature related to the risk analysis indicators, and a third target feature related to the credit analysis indicators; the analysis models include: an access analysis model, a risk analysis model, and a credit scoring model; and the data analysis results include: access probability, risk score, and credit score.
[0098] Optionally, based on the data analysis results of the target company, a decision-making plan for the target company is generated, including: Determine the weights for the probability of admission, the risk score, and the credit score; Based on the weights of the admission probability, risk score, and credit score, as well as the admission probability, risk score, and credit score, a decision-making scheme for the target company is determined.
[0099] Optionally, multiple target features are input into a pre-built analysis model to obtain the data analysis results of the target enterprise output by the analysis model, including: The first target feature is input into the admission analysis model to obtain the admission probability of the target enterprise output by the admission analysis model; If the admission probability is greater than or equal to the preset probability threshold, the second target feature is input into the risk analysis model to obtain the risk score of the target enterprise output by the risk analysis model. If the risk score is less than or equal to the preset scoring threshold, the third target feature is input into the credit scoring model to obtain the credit score of the target enterprise output by the credit scoring model.
[0100] Optionally, the multimodal data includes: public opinion data related to the target company and credit data of the target company. The multimodal data is processed, including: Identify sensitive information in credit data and perform de-identification processing on the sensitive information; Sentiment recognition is performed on public opinion data to obtain the sentiment recognition results.
[0101] Optionally, the training process of the analysis model includes: Obtain multimodal data of sample enterprises and determine the labels of data analysis results for sample enterprises; Process the multimodal data of the samples to construct a comprehensive profile of the sample enterprises; Feature extraction is performed on the panoramic portrait of the sample enterprises to obtain multiple sample candidate features of the sample enterprises, and multiple sample target features are determined from the multiple sample candidate features; Using multiple target features as training samples and the data analysis result labels of sample enterprises as sample labels, an initial analysis model is trained. After training, the analysis model is obtained.
[0102] It should be noted that the decision-making device based on multimodal data provided in this embodiment of the invention can implement all the method steps implemented in the above-mentioned decision-making method embodiment based on multimodal data, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.
[0103] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a decision-making method based on multimodal data. This method includes: acquiring multimodal data of a target enterprise; processing the multimodal data to construct a panoramic profile of the target enterprise; extracting features from the panoramic profile of the target enterprise to obtain multiple candidate features of the target enterprise; determining multiple target features from the multiple candidate features; inputting the multiple target features into a pre-constructed analysis model to obtain the data analysis results of the target enterprise output by the analysis model; the analysis model is used to analyze the multiple target features based on preset analysis indicators; and generating a decision-making scheme for the target enterprise based on the data analysis results of the target enterprise.
[0104] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for decision making based on multi-modal data, the method comprising: The method comprises the following steps: acquiring multi-modal data of a target enterprise; processing the multi-modal data to construct a panoramic portrait of the target enterprise; extracting features from the panoramic portrait of the target enterprise to obtain a plurality of candidate features of the target enterprise, and determining a plurality of target features from the plurality of candidate features; inputting the plurality of target features into a pre-constructed analysis model to obtain a data analysis result of the target enterprise output by the analysis model; the analysis model is used to analyze the plurality of target features based on a pre-set analysis index; generating a decision scheme for the target enterprise based on the data analysis result of the target enterprise.
2. The multi-modal data based decision making method of claim 1, wherein, The determination of the plurality of target features from the plurality of candidate features comprises: constructing an intervention variable of each candidate feature, and calculating a causal effect value of each candidate feature and the analysis index; determining a plurality of target features from the plurality of candidate features according to the causal effect value of each candidate feature and the data analysis index.
3. The method of claim 2, wherein, The construction of the intervention variable of each candidate feature and the calculation of the causal effect value of each candidate feature and the analysis index comprise: inputting each candidate feature into a pre-constructed strategy optimization model to obtain an intervention variable of each candidate feature output by the strategy optimization model; calculating a change amount of the analysis index corresponding to each candidate feature based on the intervention variable of each candidate feature; calculating a causal effect value of each candidate feature and the analysis index according to the intervention variable of each candidate feature and the change amount of the analysis index corresponding to each candidate feature. The strategy optimization model is trained based on sample states, sample actions and a reward function, the sample states include a plurality of sample candidate features, the sample actions include sample intervention variables of the plurality of sample candidate features, and the reward function is constructed based on a change amount of the analysis index.
4. The multi-modal data based decision making method of claim 1, wherein, The analysis index includes an admission analysis index, a risk analysis index and a credit analysis index; the plurality of target features include a first target feature related to the admission analysis index, a second target feature related to the risk analysis index and a third target feature related to the credit analysis index; the analysis model includes an admission analysis model, a risk analysis model and a credit scoring model; and the data analysis result includes an admission probability, a risk score and a credit score.
5. The method of claim 4, wherein, The generation of the decision scheme for the target enterprise based on the data analysis result of the target enterprise comprises: determining weights of the admission probability, the risk score and the credit score; determining the decision scheme for the target enterprise based on the weights of the admission probability, the risk score and the credit score, and the admission probability, the risk score and the credit score.
6. The method of claim 4, wherein, The inputting of the plurality of target features into the pre-constructed analysis model to obtain the data analysis result of the target enterprise output by the analysis model comprises: inputting the first target feature into the admission analysis model to obtain an admission probability of the target enterprise output by the admission analysis model; In a case where the access probability is greater than or equal to a preset probability threshold, the second target feature is input into the risk analysis model, and a risk score of the target enterprise output by the risk analysis model is obtained; In a case where the risk score is less than or equal to a preset score threshold, the third target feature is input into the credit score model, and a credit score of the target enterprise output by the credit score model is obtained.
7. The multi-modal data based decision making method according to any one of claims 1-6, characterized in that, The multi-modal data includes public opinion data related to the target enterprise and credit data of the target enterprise, and the processing of the multi-modal data includes: Sensitive information in the credit data is identified, and the sensitive information is desensitized; Sentiment recognition is performed on the public opinion data, and a sentiment recognition result of the public opinion data is obtained.
8. The multi-modal data based decision making method of claim 1, wherein, The training process of the analysis model includes: Sample multi-modal data of a sample enterprise is obtained, and a data analysis result label of the sample enterprise is determined; The sample multi-modal data is processed, and a sample panoramic portrait of the sample enterprise is constructed; Feature extraction is performed on the sample panoramic portrait of the sample enterprise, a plurality of sample candidate features of the sample enterprise are obtained, and a plurality of sample target features are determined from the plurality of sample candidate features; The plurality of sample target features are used as training samples, the data analysis result label of the sample enterprise is used as a sample label, an initial analysis model is trained, and after the training is completed, the analysis model is obtained. 9.A decision apparatus based on multi-modal data, characterized in that, includes: An acquisition unit is configured to acquire multi-modal data of a target enterprise; A data processing unit is configured to process the multi-modal data and construct a panoramic portrait of the target enterprise; A feature extraction unit is configured to perform feature extraction on the panoramic portrait of the target enterprise, obtain a plurality of candidate features of the target enterprise, and determine a plurality of target features from the plurality of candidate features; A data analysis unit is configured to input the plurality of target features into a pre-constructed analysis model and obtain a data analysis result of the target enterprise output by the analysis model; The analysis model is configured to analyze the plurality of target features based on a preset analysis index; A decision unit is configured to generate a decision scheme of the target enterprise based on the data analysis result of the target enterprise.
10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the multi-modal data-based decision method of any one of claims 1 to 8.