A decision information generation method and system based on multi-objective reinforcement learning

Abstract

The application discloses a kind of decision information generation method and system based on multi-objective reinforcement learning, applied to computer information processing field, comprising: obtaining and using classification model to process multidimensional information flow data, to determine the importance and timeliness of information unit;Using correlation analysis and clustering algorithm, generate and meet the information subset of the coverage degree associated with the preset key business elements;Using adaptive weight adjustment mechanism, update weight according to the real-time change of business scene and sort information unit;Through decision template, structured decision information is generated;Finally, according to the deviation of decision information and actual business result, the parameters of classification model and weight adjustment mechanism are optimized through reinforcement learning feedback mechanism, the application solves the problem that the prior art relies on static rules, poor adaptability and difficult to balance multi-objective conflict, by constructing closed-loop self-evolution system, the quality, timeliness and dynamic adaptability of decision information generation are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of computer information processing technology, and in particular to a method and system for generating decision information based on multi-objective reinforcement learning. Background Technology

[0002] In decision support systems operating in complex and dynamic environments, information processing is crucial. Existing technologies have significant limitations in generating decision information, particularly when processing massive, multi-dimensional real-time information streams.

[0003] One existing technology relies on static rules or fixed weight models to set fixed evaluation criteria for multiple dimensions such as the importance, timeliness, and relevance of information. This method cannot dynamically adjust its information processing strategy when faced with rapidly changing business scenarios, such as sudden events in the financial market or unexpected supply chain disruptions. This results in unstable quality of decision-making information output, sometimes omitting key content and sometimes responding slowly.

[0004] Another existing technology, when dealing with multi-objective optimization problems such as relevance, timeliness, and relevance, typically employs a simple trade-off strategy or focuses on a single objective. However, these objectives often have inherent conflicts; excessive pursuit of timeliness may sacrifice the depth and relevance of content, and vice versa. Existing technologies struggle to intelligently find the dynamic balance point between these conflicting objectives in different scenarios, thus the generated decision-making information often suffers from being one-sided or lacking in timeliness.

[0005] Furthermore, traditional information processing workflows are typically open-loop structures. Information is directly output after a series of fixed processing steps, but the system itself cannot adjust and optimize the front-end processing flow based on the actual application effect of the final output information. This model lacks the ability to learn from experience and improve itself, making it difficult for its system performance to achieve continuous improvement.

[0006] Therefore, existing technologies have significant shortcomings in terms of dynamic scenario adaptability, multi-objective balancing ability, and system self-optimization ability, making it difficult to meet the demand for high-quality decision information in modern complex decision-making environments. To solve these problems, there is an urgent need for a new method for generating decision information that can autonomously, intelligently, and dynamically balance multiple objectives and possess end-to-end closed-loop learning and evolution capabilities.

[0007] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] In view of this, the present invention provides a decision information generation method and system based on multi-objective reinforcement learning to solve the problems of existing technologies in generating decision information, such as the difficulty in dynamically adjusting multiple objectives such as relevance, timeliness, and relevance according to changes in real-time business scenarios, resulting in unstable decision information quality and the inability to achieve end-to-end closed-loop self-optimization. The aim is to provide a decision information generation scheme that can dynamically balance multiple objectives and has self-learning capabilities.

[0009] This invention provides a method for generating decision information based on multi-objective reinforcement learning, the method being executed by a computer device, including:

[0010] Acquire multidimensional information flow data containing timestamps and business tags, and use a classification model to process the information units in the information flow data to determine their importance score and timeliness weight;

[0011] For information units whose importance score is higher than the preset first threshold and whose timeliness weight meets the preset business scenario requirements, an association analysis algorithm and a clustering algorithm are used to process them in order to generate an information subset that is associated with the preset key business elements and meets the preset coverage.

[0012] An adaptive weight adjustment mechanism is adopted to update the importance weight and timeliness weight of each information unit in the information subset according to the real-time changes in the business scenario, and to sort the information units.

[0013] Based on the sorting results, structured decision information is generated using a pre-set decision template;

[0014] Based on the discrepancy between structured decision information and actual business scenarios, the parameters in the classification model and adaptive weight adjustment mechanism are adjusted through a feedback mechanism.

[0015] In some alternative embodiments, the feedback mechanism is implemented by a collaborative multi-agent reinforcement learning architecture.

[0016] In some alternative embodiments, the collaborative multi-agent reinforcement learning architecture includes:

[0017] A strategic intelligent agent is configured to: execute a first action, adjusting the hyperparameters of a classification model or association analysis algorithm, based on a first state formed by the long-term application effects of decision information; and

[0018] The tactical agent is configured to perform a second action that adjusts the importance weights and timeliness weights used for ranking, based on the second state formed by the real-time changes in the business scenario.

[0019] In some optional embodiments, the method further includes:

[0020] Based on the long-term application effects of biases and / or decision information, a reward value is calculated, and the reward value is used to update the strategies of the strategic agent and the tactical agent respectively.

[0021] In some optional embodiments, the association analysis algorithm is a hybrid association analysis algorithm, which includes:

[0022] Employing topic modeling to uncover topical relationships between information units; and

[0023] A word embedding model is used to calculate the semantic similarity between information units in order to uncover the semantic associations between them.

[0024] In some optional embodiments, the step of generating an information subset that satisfies a preset coverage includes:

[0025] Clustering algorithms are used to group information units, and the information set with the highest matching degree with key business elements is extracted from each group;

[0026] In response to the information set having a lower coverage of key business elements than the preset coverage, an iterative query mechanism is used to supplement the missing information and generate an information subset.

[0027] In some optional embodiments, the method further includes, before the iterative query mechanism supplements the missing information:

[0028] Representing key business elements as target semantic vectors; and

[0029] The semantic gap between the two is analyzed by calculating the vector distance between the vector representation of the information set and the target semantic vector.

[0030] The iterative query mechanism generates supplementary query instructions based on the semantic gap to obtain supplementary information.

[0031] In some optional embodiments, the method further includes, before generating structured decision information:

[0032] Based on the sorting results, filter information units from high to low;

[0033] If the similarity between the selected information units is higher than the preset deduplication threshold, the information units with lower rankings will be removed using a deduplication algorithm.

[0034] In some optional embodiments, the adaptive weight adjustment mechanism is triggered based on real-time changes in the business scenario to increase or decrease the importance weight and / or timeliness weight of specific information units related to real-time changes.

[0035] In some optional embodiments, the method further includes:

[0036] In the memory, the semantic and logical relationships between information units mined through association analysis algorithms are used to construct an associated information network data structure.

[0037] This invention provides a decision information generation system based on multi-objective reinforcement learning, comprising:

[0038] The information processing module is configured to acquire multidimensional information flow data containing timestamps and business tags, and use a classification model to process the information units in the information flow data to determine their importance score and timeliness weight.

[0039] The subset generation module is configured to process information units with importance scores higher than a preset first threshold and timeliness weights that meet the requirements of preset business scenarios using association analysis algorithms and clustering algorithms to generate information subsets that are associated with preset key business elements and meet preset coverage.

[0040] The sorting module is configured to use an adaptive weight adjustment mechanism to update the importance weight and timeliness weight of each information unit in the information subset according to the real-time changes in the business scenario, and sort the information units.

[0041] The decision generation module is configured to generate structured decision information based on the sorting results from the sorting module, using a preset decision template; and

[0042] The feedback adjustment module is configured to adjust the parameters in the classification model and adaptive weight adjustment mechanism based on the deviation between the structured decision information and the actual business scenario through a feedback mechanism.

[0043] In some optional embodiments, the feedback adjustment module is further configured to execute the feedback mechanism through a cooperative multi-agent reinforcement learning architecture.

[0044] In some alternative embodiments, the collaborative multi-agent reinforcement learning architecture includes:

[0045] The strategic intelligent agent is configured to: perform a first action to adjust the hyperparameters of the classification model or association analysis algorithm based on the first state formed by the long-term application effect of the decision information;

[0046] The tactical agent is configured to perform a second action that adjusts the importance weights and timeliness weights used for ranking, based on the second state formed by the real-time changes in the business scenario.

[0047] In some optional embodiments, the subset generation module is further configured to:

[0048] Clustering algorithms are used to group information units, and the information sets with the highest matching degree with key business elements are extracted from each group; and

[0049] In response to the information set having a lower coverage of key business elements than the preset coverage, an iterative query mechanism is used to supplement the missing information and generate an information subset.

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention.

[0051] The decision information generation method and system based on multi-objective reinforcement learning of the present invention have the following beneficial effects:

[0052] The decision information generation method provided by this invention, through reinforcement learning, enables the system to autonomously adjust its information processing strategy based on real-time business scenarios, thereby improving the quality of decision information in complex environments. By intelligently optimizing multiple objectives such as balance, timeliness, and relevance, the generated decision information possesses depth, timeliness, and comprehensiveness. A complete closed loop from information generation to effect feedback is constructed, allowing the system to reflect and learn based on actual application results, continuously iterating and optimizing internal models and parameters to improve system performance. Through comprehensive metric evaluation, semantic gap analysis, and targeted supplementation, the information foundation for decision-making is ensured to be more comprehensive and in-depth, reducing biases caused by information omissions. By using tactical intelligent agents to capture business signals and adjust prioritization strategies, urgent information is ensured to receive the highest priority, shortening decision delays. Attached Figure Description

[0053] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0054] Figure 1 This is a flowchart of a decision information generation method based on multi-objective reinforcement learning according to an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the structure of a decision information generation system based on multi-objective reinforcement learning according to an embodiment of the present invention. Detailed Implementation

[0056] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0057] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0058] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined. Therefore, the actual execution order may change depending on the specific circumstances.

[0059] In the field of computer information processing, information fusion technology integrates and analyzes multi-source heterogeneous data to extract the correlations and patterns hidden in complex data, providing support for decision-making. Classification models automatically categorize input information units based on a predefined category system, enabling rapid filtering and organization of massive amounts of information. Association analysis algorithms are used to uncover potential connections between information units, construct information networks, and reveal dependencies between information. Clustering algorithms group similar information units into the same group, forming different information clusters, facilitating the understanding of the overall information structure. Reinforcement learning optimizes decision-making strategies through interactive learning between the agent and the environment, achieving adaptive control of complex dynamic systems. This solution utilizes information fusion technology to integrate multi-source information, uses classification models for initial screening and organization of information units, employs association analysis and clustering algorithms to deeply mine relationships between information, and utilizes reinforcement learning to dynamically optimize each stage of decision information generation, thereby improving the efficiency and quality of decision information generation.

[0060] like Figure 1 As shown, this embodiment of the invention provides a decision information generation method based on multi-objective reinforcement learning. This method is applied in the field of computer information processing technology and is executed by a computer device. It aims to solve the problems of poor adaptability of decision information generation systems in complex dynamic environments, difficulty in balancing multi-objective conflicts, and the lack of self-optimization in open-loop processing flows. It includes the following steps:

[0061] S100. Execution of Information Acquisition and Preliminary Processing Steps: The computer device acquires multi-dimensional information flow data containing timestamps and business tags from multiple data sources. For example, news data is acquired from a financial news API; each piece of data includes a news title, body text, publication timestamp, and business tags identifying the news type, such as "company performance" or "macroeconomic policy." Simultaneously, the device also acquires post data related to specific industries or companies from social media platforms; each piece of data includes text content and a publication timestamp. This data is uniformly considered as information units. Then, the computer device uses a classification model to process each information unit to determine its relevance score and timeliness weight. This classification model can be a Transformer-based natural language processing model, such as the BERT model, which has been pre-trained using a large amount of financial domain corpus. For each input information unit, the classification model outputs a relevance score for that information unit, ranging from 0 to 1, with higher values ​​indicating a more reliable information unit. The classification model also outputs a business tag for the information unit, such as "mergers and acquisitions," "personnel changes," or "regulatory policy." To determine the timeliness weight of an information unit, the difference between the information unit's timestamp and the current system time is first calculated, in hours. Then, an exponential decay function is used to calculate the timeliness weight. This function can be expressed as: ,in It is the attenuation coefficient. It's the time difference. Attenuation coefficient. It can be adjusted according to specific business needs.

[0062] Alternatively, the classification model can use other types of natural language processing models, such as recurrent neural networks like LSTM or GRU.

[0063] S200, Execution of Information Subset Generation Steps: For information units with a targeting score higher than a preset first threshold and a timeliness weight meeting the requirements of a preset business scenario, an association analysis algorithm and a clustering algorithm are used for processing to generate an information subset associated with a preset business element and meeting a preset coverage. The preset first threshold is used to filter out information units with low targeting, and can be set to 0.6. The preset business scenario requirements can be adjusted according to specific business needs. For example, in a financial risk monitoring scenario, a timeliness weight higher than 0.4 may be required. The association analysis algorithm is used to mine the relationships between information units. In this embodiment, the association analysis algorithm can adopt a hybrid association analysis method, including topic association analysis based on topic models and semantic association analysis based on word embedding models. Topic association analysis uses the LDA topic model to extract topics from information units, considering information units belonging to the same topic as having topic associations. Semantic association analysis uses pre-trained word embedding models, such as Word2Vec or GloVe, to calculate the semantic similarity between information units, considering information units with semantic similarity higher than a certain threshold as having semantic associations. The clustering algorithm is used to divide related information units into different clusters. In this embodiment, the clustering algorithm can employ the Louvain algorithm, a community detection algorithm based on modularity optimization. This algorithm can group closely related information units into the same cluster. Business elements represent the topics or issues that decision-makers are most concerned about, such as "liquidity risk of Company XX". Business elements can be represented as semantic vectors, and the degree of association between a cluster and a business element is measured by calculating the cosine similarity between the average semantic vector of each cluster and the semantic vector of the business element. Coverage is used to measure the degree to which a subset of information covers business elements. If the coverage of a subset of information on business elements is lower than a preset coverage, an iterative query mechanism is needed to supplement the missing information to improve the coverage. The preset coverage can be set to 0.8. The iterative query mechanism first analyzes the semantic gap between the current subset of information and the business elements, and then generates a supplementary query instruction based on the semantic gap to obtain the missing information from external data sources and add it to the subset of information. For example, if the business element is "liquidity risk of Company XX", and the current subset of information lacks information about "balance sheet of Company XX", a query instruction "balance sheet of Company XX" can be generated to obtain relevant information from financial data websites or databases.

[0064] Alternatively, other types of association rule mining algorithms can be used for association analysis, such as the Apriori algorithm or the FP-Growth algorithm. Other types of clustering algorithms can also be used, such as the K-means algorithm or hierarchical clustering algorithms.

[0065] S300. Execute the adaptive weight adjustment and ranking steps: An adaptive weight adjustment mechanism is adopted to update the attribute weight and timeliness weight of each information unit in the information subset according to the real-time changes in the business scenario, and to rank the information units. The adaptive weight adjustment mechanism is used to dynamically adjust the attribute weight and timeliness weight of information units according to the real-time changes in the business scenario. In this embodiment, the adaptive weight adjustment mechanism is implemented through a tactical intelligent agent. The tactical intelligent agent monitors the real-time data stream of the business scenario, such as stock prices and market sentiment. When a significant change in the business scenario is detected, the tactical intelligent agent will perform an action to adjust the attribute weight and timeliness weight of the information units related to the change. For example, when it is detected that the stock price of "XX Company" has fallen sharply in a short period of time, the tactical intelligent agent will increase the attribute weight and timeliness weight of information units related to "XX Company" so that these information units can be given priority by decision-makers. The comprehensive score of an information unit is obtained by weighted summation of its attribute weight and timeliness weight. The information units are ranked according to their comprehensive scores, with information units with higher scores ranking higher.

[0066] Alternatively, the adaptive weight adjustment mechanism can employ other types of machine learning algorithms, such as decision trees or support vector machines.

[0067] S400, Execution Decision Information Generation Step: Based on the sorting results, a structured decision information is generated using a preset decision template. The decision template defines the format and content of the decision information. In this embodiment, the decision template can be a JSON format template, containing fields such as risk topic, risk level, and information. The system fills the relevant information into the decision template according to the sorted information units to generate structured decision information. For example, if the risk topic is "liquidity risk of XX company," the risk level is "high," and the information includes news reports from social media platforms such as XX News, the generated decision information could be as follows:

[0068] json

[0069] { "risk_subject": "XX Company's liquidity risk", "risk_level": "High", "key_infos": [ {"source": "XX Company", "content": "..."}, {"source": "XX Company", "content":"..."} ]

[0070] }

[0071] ```

[0072] S500, Execution Feedback Adjustment Step: Based on the deviation between structured decision information and an actual business scenario, the parameters in the classification model and adaptive weight adjustment mechanism are adjusted through a feedback mechanism. The feedback mechanism is used to adjust the parameters in the classification model and adaptive weight adjustment mechanism according to the actual application effect of the decision information, thereby improving the quality of the decision information. In this embodiment, the feedback mechanism is implemented through a collaborative multi-agent reinforcement learning architecture, which includes a strategic agent and a tactical agent. The strategic agent is responsible for adjusting the parameters of the classification model and association analysis algorithm, while the tactical agent is responsible for adjusting the parameters of the adaptive weight adjustment mechanism. The strategic agent adjusts the hyperparameters of the classification model and association analysis algorithm based on the long-term application effect of the decision information, such as the accuracy and recall rate of risk warnings, to improve their performance. The tactical agent adjusts the parameters of the adaptive weight adjustment mechanism based on real-time feedback from the business scenario, such as changes in stock prices and market sentiment, to better adapt to changes in the business scenario. Deviation is used to measure the difference between the decision information and the actual business scenario. For example, if the decision information predicts that "Company XX" has liquidity risk, but the actual situation is that "Company XX" has not experienced a liquidity crisis, then there is a deviation. The system calculates reward values ​​based on deviations and feeds these rewards back to the strategic and tactical agents. The strategic and tactical agents then update their respective strategies based on the reward values ​​to improve the quality of future decisions.

[0073] Alternatively, the feedback mechanism can employ other types of optimization algorithms, such as gradient descent or genetic algorithms.

[0074] Through the above steps, this embodiment of the invention can autonomously learn and adjust information processing strategies according to real-time business scenarios, achieving dynamic adaptive decision-making. Furthermore, this embodiment of the invention intelligently seeks a dynamic balance point among multiple objectives based on different scenarios, rather than simply making static trade-offs, ensuring that the final generated decision information is in-depth, timely, and comprehensive. This embodiment of the invention constructs a closed loop from information generation to effect feedback; the system learns based on actual application results, continuously iterating and optimizing all models and parameters within the system.

[0075] In some embodiments, the feedback mechanism is implemented by a collaborative multi-agent reinforcement learning architecture. This architecture is a component of the present invention's ability to achieve closed-loop self-learning and evolution, used to adjust parameters in the classification model and adaptive weight adjustment mechanism, thereby improving the quality and adaptability of decision information. Specifically, this collaborative multi-agent reinforcement learning architecture comprises two agents: a strategic agent and a tactical agent, deployed on specific computing nodes in a server cluster. The two agents share the underlying infrastructure and data interfaces, but possess their own independent policy networks and reward functions, each responsible for parameter adjustment tasks at different time scales, and collaborate to optimize the overall system performance.

[0076] The strategic agent is responsible for tuning the hyperparameters of underlying models, such as classification models and association analysis algorithms. In the scenario of financial market risk monitoring, the strategic agent may need to adjust hyperparameters such as the learning rate, dropout rate, and number of attention layers of the BERT model, or adjust the number of topics in the LDA topic model. The strategic agent uses the long-term application effect of decision information as its state input, such as the accuracy, recall, and impact value on actual business of risk warning reports generated in the past week. These long-term indicators constitute a multi-dimensional state space, which the strategic agent observes to evaluate the overall performance of the current system. The strategic agent uses a deep Q-network algorithm to learn strategies, and the action space includes a series of hyperparameter tuning schemes. For example, actions might include "increasing the learning rate of the BERT model by 0.0001" or "reducing the number of topics in the LDA model by 5." Based on the current state, the strategic agent selects an action and translates it into configuration instructions for the underlying models. These instructions are sent to the corresponding model training modules via API interfaces, triggering model retraining or parameter updates. The goal of a strategic intelligent agent is to maximize long-term cumulative rewards, that is, to improve the overall performance and stability of the system over a longer period of time. The design of the reward function will take into account multiple long-term indicators, such as warning accuracy, recall rate, and potential losses avoided.

[0077] The tactical agent is responsible for adjusting the gender and timeliness weights of information units in real time. The tactical agent uses real-time changes in the business scenario as its state input, such as stock price fluctuations, breaking news events, and changes in public opinion on social media. These real-time signals are pushed to the tactical agent in real time through data subscription and event monitoring mechanisms. The tactical agent also uses a deep Q-network algorithm to learn strategies, and its action space includes a series of adjustment schemes for weight values. For example, actions might include "increasing the gender weight of information units related to XX company by 1.5 times" or "increasing the timeliness weight of all information units by 2 times." Based on the current state, the tactical agent selects an action and translates it into an instruction to adjust the weight values ​​of information units. These instructions directly affect the ranking module, influencing the final ranking result of the information units. The goal of the tactical agent is to maximize immediate rewards, that is, to improve the quality and response speed of decision-making information in a short period of time. The design of the reward function will focus on real-time indicators, such as decision delay and whether information is captured in a timely manner.

[0078] Two intelligent agents achieve collaboration through sharing state information and cooperative reward mechanisms. For example, a strategic agent can decompose long-term reward signals into guidance information for a tactical agent, helping the tactical agent learn the optimal strategy more quickly. Conversely, the short-term behavior of the tactical agent also affects the long-term reward of the strategic agent, thus forming a closed loop of mutual influence and co-evolution.

[0079] Optionally, the collaborative multi-agent reinforcement learning architecture can employ other reinforcement learning algorithms, such as A3C and PPO. Furthermore, the design of the agent's state space, action space, and reward function can be adjusted according to the specific application scenario. For example, in a supply chain risk monitoring scenario, the state input of the tactical agent might include supplier financial data, real-time logistics and transportation status, and weather forecasts, while the action space might include adjustments to warning thresholds for different risk levels.

[0080] Through the above technical solutions, a feedback mechanism based on collaborative multi-agent reinforcement learning is realized, enabling the system to dynamically adjust the parameters in the classification model and adaptive weight adjustment mechanism according to the deviation of the actual business scenario, thereby improving the quality and adaptability of decision information and realizing end-to-end closed-loop self-evolution.

[0081] In some embodiments, the cooperative multi-agent reinforcement learning architecture includes strategic agents and tactical agents, as specifically implemented below:

[0082] The strategic agent is configured to perform the first action of adjusting the hyperparameters of the classification model or association analysis algorithm, based on the first state constituted by the long-term application effect of the decision information. The long-term application effect can be quantified as follows: the system can record the changes in relevant business indicators over a period of time, such as one month, after each piece of decision information is generated. For example, in financial market risk monitoring applications, it can record the actual amount of loss incurred after risk control measures are taken based on the decision information. The lower the loss amount, the higher the quality of the decision information and the better the long-term application effect. The system can use a sliding window to statistically analyze these long-term indicators, forming a time series. This time series can serve as the first state of the strategic agent. The strategic agent's first action is to adjust the hyperparameters of the classification model, such as the BERT model, or the association analysis algorithm, such as the LDA topic model. For the BERT model, adjustable hyperparameters include the learning rate, the number of training epochs, and the dropout rate. For example, the strategic agent can decide to adjust the learning rate of the BERT model from 0.001 to 0.0005 to improve the model's generalization ability. For the LDA topic model, adjustable hyperparameters include the number of topics. For example, a strategic agent might decide to increase the number of LDA topic models from 20 to 25 to more finely characterize the topic distribution of information units. The strategic agent can use deep reinforcement learning algorithms such as Deep Q-Networks (DQNs) to learn its policy—that is, what hyperparameter adjustments should be made in a given first state to maximize long-term rewards. Optionally, in addition to adjusting hyperparameters, the strategic agent can also adjust the structure of the classification model or association analysis algorithm. For example, the strategic agent might decide to increase the number of layers in the BERT model or replace the LDA model with another topic model, such as Non-negative Matrix Factorization (NMF).

[0083] The tactical agent is configured to perform a second action—adjusting the gender and timeliness weights used for ranking—based on a second state comprised of real-time changes in the business scenario. These real-time changes can be captured by monitoring external data streams. For example, in financial market risk monitoring applications, data streams such as stock prices, trading volumes, and news sentiment can be monitored. Significant fluctuations in stock prices or the emergence of sudden negative news indicate a significant change in the business scenario. This real-time data can serve as the tactical agent's second state. The agent's second action is to adjust the gender and timeliness weights used for ranking. For example, when a company's stock price is detected to have fallen sharply in a short period, the agent can decide to increase the gender weight of information units related to that company by a factor of 2 and the timeliness weight by a factor of 1.5 to ensure that this information is prioritized for decision-makers. The agent can use reinforcement learning algorithms, such as policy gradient algorithms, to learn its policy—that is, what weight adjustment action should be taken in a given second state to maximize immediate rewards.

[0084] Strategic and tactical intelligent agents collaborate to optimize the decision-making information generation system. The strategic agent is responsible for long-term optimization, improving the overall system performance; the tactical agent is responsible for short-term tuning, quickly responding to changes in business scenarios. The two agents collaborate by sharing reward signals. The calculation method for these reward signals can be determined based on the actual application effect of the decision information. For example, if the decision information successfully warns of a risk event, a positive reward can be given; if the decision information fails to warn in a timely manner or gives an incorrect warning, a negative reward can be given. The two agents update their strategies based on their respective reward signals, thereby achieving co-evolution.

[0085] Through the above methods, this embodiment achieves dynamic optimization of the classification model, association analysis algorithm, and ranking weight parameters, enabling the decision information generation system to better adapt to complex dynamic environments and generate high-quality decision information.

[0086] In some embodiments, the method further includes: calculating a reward value based on the deviation between the structured decision information and the actual business scenario and / or the long-term application effect of the decision information, and using the reward value to update the strategies of the strategic agent and the tactical agent respectively.

[0087] Specifically, for application scenarios of financial market risk monitoring, the implementation process of this technical solution is explained in detail.

[0088] 1. Deviation Calculation: In step S500 of the above embodiment, the system has compared the generated structured decision information with the actual business results 24 hours later and calculated the deviation. Deviation can be quantified in various ways. For example, if the decision information predicts "XX Company has a high risk of debt default" and the actual result is "XX Company issues a debt default announcement", then the deviation can be set to 0, indicating a perfect match; if the decision information predicts "XX Company has a moderate risk of debt default" and the actual result is "XX Company issues a debt default announcement", then the deviation can be set to 0.5, indicating a partial match; if the decision information predicts "XX Company has a low risk of debt default" and the actual result is "XX Company issues a debt default announcement", then the deviation can be set to 1, indicating a complete mismatch.

[0089] Optionally, the calculation of Deviation can employ more complex models, such as considering the gap between the predicted risk level and the actual risk level, and assigning different weights to different types of risk level differences.

[0090] 2. Long-term Application Effectiveness Evaluation: The system can track the application effectiveness of the generated decision-making information over a long period. For example, it can statistically analyze metrics such as the accuracy and recall rate of high-risk warnings generated by the system over the past month. These metrics can comprehensively reflect the overall performance and decision-making quality of the system. Long-term application effectiveness can be quantified into multiple dimensions of metrics, such as accuracy, recall rate, F1 score, and average warning time. Optionally, the system can also incorporate expert evaluation, inviting domain experts to score the quality of the decision-making information, and using expert scores as a supplement to the long-term application effectiveness.

[0091] 3. Reward Calculation: The system calculates the reward value based on deviation and long-term application performance indicators. The design of the reward function is crucial and needs to be adjusted according to actual business objectives. One possible reward function is as follows:

[0092]

[0093] in, , , These are weighting coefficients, representing the weights for bias, accuracy, and... The degree of importance attached to it. Accuracy represents the level of precision. The F1 score represents the harmonic mean of precision and recall, used to comprehensively evaluate the system's precision and recall in identifying risky events. For example, if deviation is 0, accuracy is 0.9, F1 score is 0.8, and α=0.5, β=0.3, γ=0.2, then Reward = 0.5 * (1 - 0) + 0.3 * 0.9 + 0.2 * 0.8 = 0.5 + 0.27 + 0.16 = 0.93.

[0094] Optionally, the reward function can be more complex, for example, by considering the different types of risk warnings and assigning higher rewards to successful warnings of high-risk events.

[0095] Optionally, the reward function may also include a penalty for delayed alerts; for example, if the system issues an alert only after a risk event has occurred, the reward value will be reduced accordingly.

[0096] 4. Strategy Update: The system uses the calculated reward value to update the strategies of both the strategic and tactical agents. a. Tactical Agent Strategy Update: The tactical agent optimizes its short-term parameter adjustment strategy based on immediate rewards. The goal of the tactical agent is to maximize short-term rewards, i.e., to respond to real-time business events as quickly as possible and generate accurate decision-making information. The tactical agent can use reinforcement learning algorithms such as Q-learning and SARSA for strategy updates. For example, if the tactical agent increases the timeliness and relevance weights of relevant information units by 1.5 times and 2 times respectively when "XX company's stock price fluctuates abnormally," and ultimately generates an accurate risk warning, obtaining a higher reward value, then the Q-learning algorithm will update the tactical agent's Q-value table, making the tactical agent more inclined to take similar actions under similar circumstances. The specific Q-value update formula is as follows:

[0097]

[0098] in, Representing the current state, "abnormal stock price fluctuation of Company XX" is one such state. An action representing the actions taken by a tactical intelligent agent, such as "increasing the sex weight and timeliness weight of relevant information units by 1.5 times and 2 times respectively," is one such action. Represents the reward value. Represents the learning rate. Represents the discount factor. Represents the next state. This represents the next action. Optionally, the tactical agent can also employ deep reinforcement learning algorithms, such as Deep Q-Network (DQN), which uses neural networks to approximate the Q-value function, thereby handling more complex state and action spaces. b. Strategic Agent Policy Update: The strategic agent optimizes the system's underlying model and algorithm parameters based on long-term rewards. The strategic agent's goal is to maximize long-term cumulative rewards, fundamentally improving the system's overall performance and decision-making quality. The strategic agent can use policy gradient algorithms, evolutionary algorithms, etc., for policy updates. For example, if the strategic agent discovers in the past week that increasing the number of hidden layers in the classification model improves the system's overall accuracy, the policy gradient algorithm will update the strategic agent's policy network, making the strategic agent more inclined to make similar adjustments.

[0099] Optionally, the strategic agent can also employ a Bayesian optimization algorithm to search for the optimal combination of hyperparameters.

[0100] This embodiment incorporates biases and long-term application effects into the feedback loop and utilizes reinforcement learning algorithms to continuously optimize the strategies of strategic and tactical agents. The system can continuously learn and evolve on its own, thereby continuously generating high-quality decision-making information in complex dynamic environments and improving the accuracy and timeliness of decision-making.

[0101] In some embodiments, the association analysis algorithm can be a hybrid association analysis algorithm that integrates topic modeling and word embedding modeling to achieve the mining of deep associations between information units.

[0102] Specifically, for mining topic associations, a Latent Dirichlet (LDD) distribution model can be used. First, the pre-processed set of information unit texts is input into a pre-trained LDD topic model. This model takes the text content of the information units as input, analyzes it, and assigns one or more topics and corresponding probability values ​​to each information unit. For example, for an information unit about "Company A releasing a new product," the model might output topics such as "Company Operations" (probability 0.7), "Technology" (probability 0.2), and "Market Competition" (probability 0.1). The system pre-extracts 20 topics, which can represent different business areas or risk types. If two information units belong to the same topic, and the probability corresponding to that topic is higher than 0.5, then the two information units are considered to have a topic association. This topic model can be trained using the Gibbs sampling algorithm, and the training corpus can use a large amount of historical news data and industry reports.

[0103] For semantic association mining, the Sentence-BERT model can be used. This model takes the text content of each information unit as input and calculates its vector representation to obtain the semantic vector for each unit. Then, it calculates the cosine similarity between any two information unit vectors. A higher cosine similarity indicates a stronger semantic similarity and association between the two information units. For example, if two information units describe "Company B's profit growth" and "Company B's performance exceeded expectations," respectively, although they may differ literally, the semantic vector similarity calculated by the Sentence-BERT model will be high, indicating a semantic association between them. If the cosine similarity of the semantic vectors of two information units is greater than 0.8, a strong semantic association is considered to exist between them. This model can use publicly available pre-trained models and be fine-tuned using financial corpora to improve its semantic representation capabilities in specific domains.

[0104] Optionally, topic models can use other topic modeling methods, such as latent semantic analysis or hierarchical Dirichlet distribution models, to adapt to different data characteristics and business needs. In other optional implementations, semantic similarity can be calculated using other distance metrics, such as Euclidean distance or Manhattan distance, but the similarity threshold needs to be adjusted according to the actual situation.

[0105] By integrating topic models and word embedding models, this hybrid association analysis algorithm can mine the connections between information units from different dimensions, taking into account both macro-level topic connections and micro-level semantic connections. This allows for a more comprehensive and accurate construction of the associated information network, providing richer information for the subsequent generation of information subsets.

[0106] In some embodiments, the computer device performs a step of generating an information subset that meets a preset coverage. This step includes two stages: information unit grouping and information set extraction, and iterative query supplementation, to ensure that the information subset can fully cover the preset business elements.

[0107] 1. Information unit grouping and information set extraction stage:

[0108] a. Specific Implementation of the Clustering Algorithm: The system employs the Louvain community detection algorithm to group information units in the associated information network. The Louvain algorithm is a clustering algorithm based on modularity optimization. Its specific steps include: i. Initialization: Treat each information unit as an independent community. ii. Modularity Optimization: For each information unit, attempt to move it to its neighboring community, calculating the change in modularity of the entire associated information network before and after the move. If the modularity increases after the move, accept the move; otherwise, cancel it. iii. Repeated Iteration: Repeat step ii until the modularity of the entire associated information network no longer increases significantly. iv. Community Merging: Treat each community as a new node, and the edge weights between communities are the sum of the edge weights between nodes within the original community. Reconstruct the associated information network based on the new nodes and edge weights. v. Iteration: Repeat steps ii to iv until the entire associated information network has only one community. Ultimately, the Louvain algorithm will divide the information units in the associated information network into several different communities. Information units within each community have high correlation, while information units between different communities have low correlation. The system treats each community as a group. Alternatively, other clustering algorithms can be used, such as K-means or spectral clustering, as long as they can divide information units into different groups. For example, the K-means algorithm can divide information units into K distinct clusters based on their semantic vectors, with each cluster containing information units exhibiting high semantic similarity.

[0109] b. Specific Implementation of Information Set Extraction: For each group, the system calculates the average semantic vector of all information units within that group, which serves as the vector representation of that group. This average semantic vector can be obtained by weighted averaging of the semantic vectors of all information units within the group, with the weights being the personality scores of the information units. The system pre-represents the business element as a target semantic vector. Then, the system calculates the cosine similarity between the average semantic vector of each group and the target semantic vector, and takes the set of information units contained in the group with the highest cosine similarity as the information set with the highest matching degree with the business element. Optionally, the system can also use other matching degree calculation methods, such as Euclidean distance, Manhattan distance, etc. For example, the system can calculate the Euclidean distance between the average semantic vector of each group and the target semantic vector, and take the set of information units contained in the group with the smallest Euclidean distance as the information set with the highest matching degree with the business element.

[0110] 2. Iterative query supplementation phase:

[0111] a. Specific Implementation of Coverage Calculation and Judgment: The system calculates the coverage of the current information set to the business elements. In this embodiment, coverage is defined as the cosine similarity between the average semantic vector of the current information set and the target semantic vector. If this cosine similarity is lower than a preset coverage threshold, the current information set is considered to have insufficient coverage of the business elements and needs to be supplemented. For example, if the preset coverage threshold is 0.85, and the cosine similarity between the average semantic vector of the current information set and the target semantic vector is 0.7, then supplementation is required. Optionally, the system can also use other coverage calculation methods, such as calculating the proportion of words related to business elements in the current information set.

[0112] b. Specific Implementation of Iterative Query Supplementation: When the coverage falls below a preset coverage threshold, the system initiates an iterative query supplementation mechanism. The specific steps of this mechanism include: i. Semantic Gap Analysis: Calculate the difference vector between the average semantic vector of the current information set and the target semantic vector. This difference vector represents the semantic gap between the current information set and the business elements. For example, if the value of the "legal proceedings" related dimension in the target semantic vector is high, while the value of the "legal proceedings" related dimension in the average semantic vector of the current information set is low, it indicates that the current information set lacks information on "legal proceedings." ii. Generating Supplementary Query Instructions: Based on the semantic gap, the system automatically generates supplementary query instructions. For example, if the semantic gap indicates that the current information set lacks information on "legal proceedings," the system can combine "legal proceedings" with the name of the business element to form a new query term, such as "XX Company legal proceedings." iii. External Query: The system sends the supplementary query instructions to an external search engine API to obtain new information units related to the query terms. iv. Information Set Update: The system adds the newly queried information units to the current information set and recalculates the coverage. v. Iteration: Repeat steps i to iv until the current information set reaches a preset coverage threshold for the business elements. Through iterative querying and supplementation, the system can continuously supplement missing information until the information set fully covers the business elements.

[0113] This embodiment ensures that the generated information subset is not only relevant to business elements but also fully covers all aspects of those elements, thus providing a comprehensive and in-depth information foundation for subsequent decision-making. This solution effectively solves the problem of information omissions caused by incomplete information and insufficient correlation mining in traditional technologies.

[0114] In some embodiments, the specific implementation process of semantic gap analysis before the iterative query mechanism supplements the missing information is as follows:

[0115] First, business elements are represented as target semantic vectors. Specifically, the system inputs pre-defined "business elements" representing the decision-maker's concerns, such as "XX Company's debt repayment ability in the next three months," into a pre-trained Sentence-BERT model. This model encodes this textual information into a 768-dimensional floating-point vector, which is the target semantic vector, and stores it in computer memory. The Sentence-BERT model is fine-tuned using cosine similarity as a similarity metric, making it more suitable for generating sentence embeddings for semantic similarity comparison and better capturing the semantic information of business elements.

[0116] Alternatively, other word embedding models, such as Word2Vec or GloVe, can be used, but it is necessary to ensure that the model is fine-tuned with a domain-specific corpus to improve its ability to represent business elements.

[0117] Secondly, the semantic gap between the information set and the target semantic vector is analyzed by calculating the vector distance between them. The system first calculates the vector representation of the information set by converting the text content of each information unit in the set into a 768-dimensional semantic vector using the Sentence-BERT model. These vectors are then averaged to obtain the average semantic vector of the information set. Next, the system calculates the cosine distance between the average semantic vector of the information set and the target semantic vector. This cosine distance quantifies the degree of difference between the two, reflecting the extent to which the information set covers business elements. A greater difference indicates lower coverage and a larger semantic gap.

[0118] Alternatively, other vector distance metrics, such as Euclidean distance or Manhattan distance, can be used, but the subsequent semantic gap analysis and supplementary query instruction generation strategies will need to be adjusted accordingly.

[0119] Finally, the iterative query mechanism generates supplementary query instructions to obtain additional information based on the semantic gap. Specifically, if the calculated cosine distance is greater than a preset distance threshold, such as 0.2, it indicates that the information set does not adequately cover the business elements, and there is a significant semantic gap. The system will analyze this semantic gap, that is, calculate the difference vector between the target semantic vector and the average semantic vector of the information set. Each dimension in this difference vector corresponds to a specific semantic direction, and the dimensions with larger values ​​correspond to semantic concepts that are missing in the information set but are crucial for understanding the business elements. The system selects the three dimensions with the largest absolute values ​​in the difference vector and uses the semantic concepts corresponding to these three dimensions as supplementary query terms. For example, if the business element is "XX Company's debt repayment ability in the next three months," but the information set lacks information about "cash flow," the generated supplementary query terms might be "XX Company" and "cash flow." The system then uses these terms to initiate supplementary queries to external search engine APIs and adds the new information units retrieved to the information set, repeating the coverage calculation and semantic gap analysis process until the coverage meets the requirements. Through this iterative query mechanism, the system can automatically fill in missing information, narrow the semantic gap, and ultimately generate more comprehensive and accurate decision-making information.

[0120] Through the above specific implementation methods, this embodiment can transform business elements into quantifiable target semantic vectors, and effectively analyze the semantic gap between the two by calculating the vector distance between the information set and the target semantic vector. This guides the iterative query mechanism to generate more targeted supplementary query instructions, achieves accurate supplementation of missing information, and improves information coverage and the accuracy of decision information.

[0121] In some embodiments, the system performs a deduplication process before generating structured decision information to ensure that the information presented to decision-makers is concise and free of redundancy.

[0122] This step first involves calculating the results from the previous steps. The information units in the information subset are sorted in descending order. Common sorting algorithms such as quicksort, mergesort, or heapsort can be used. This embodiment uses the quicksort algorithm, whose average time complexity is O(log n). ,in This refers to the number of information units.

[0123] The system then filters information units one by one, starting from the head of the sorted list. For each information unit, the system calculates its semantic similarity with subsequent information units in the list. The semantic similarity calculation uses a pre-trained Sentence-BERT model. Specifically, this model encodes the text content of two information units into 768-dimensional semantic vectors, and then calculates the cosine similarity between these two vectors. If the cosine similarity between two information units is higher than a preset deduplication threshold, they are considered highly similar semantically and constitute redundant information. In this embodiment, the preset deduplication threshold is 0.95. The system compares the two information units... The information units with higher scores are retained, while those with lower scores are removed from the list.

[0124] Alternatively, similarity can be calculated using other methods, such as Jaccard similarity or edit distance-based methods. The deduplication threshold can also be adjusted according to the specific application scenario. For example, for scenarios with low tolerance for information duplication, the deduplication threshold can be set to 0.98 or even higher.

[0125] Optionally, the deduplication algorithm doesn't simply delete information units with lower scores; instead, it merges two similar information units. For example, it can merge the text content of two information units and recalculate their scores. The system continuously performs filtering and deduplication operations until the sorted list is completely traversed. After this process, the information units in the final information list have high semantic distinctiveness, avoiding information duplication and thus improving decision-making efficiency. For example, suppose the sorted information list contains the following two information units: Information Unit 1: The source is the social media platform XX, whose... The value is 0.98, and the content is "XX Company announces a strategic cooperation agreement with YY Company." Information Unit 2: The source is another social media platform, XXX Society. The cosine similarity of information unit 1 is 0.96, and the content is "Sources say that XX Company and YY Company have signed a cooperation agreement." These two information units are highly similar semantically, with a cosine similarity higher than 0.95. Because information unit 1... If the information unit is higher than information unit 2, the system will retain information unit 1 and delete information unit 2.

[0126] Through the deduplication and filtering steps described above, the system can effectively reduce redundant information in decision-making information, improve the simplicity and effectiveness of information, thereby helping decision-makers to grasp information more quickly and improve decision-making efficiency.

[0127] In some embodiments, the adaptive weight adjustment mechanism adjusts the information unit weight and / or timeliness weight in the following ways.

[0128] The system monitors business scenario data from multiple channels in real time, such as stock trading data, news sentiment data, and social media data. This data is converted into structured event information and input into a pre-trained event detection model. This event detection model can identify events related to preset business objectives, such as abnormal trading volume fluctuations of a specific stock, outbreaks of negative news related to the company, and concentrated complaints about the company's products on social media.

[0129] When the event detection model identifies a specific event, the system triggers corresponding weight adjustments based on preset rules. For example, if it detects a significant and abnormal increase in the trading volume of a stock within a short period, the system will adjust the nature score and timeliness weight of all information units related to that stock. Specifically, the nature score of these information units can be multiplied by a coefficient greater than 1, such as 1.2, to increase their priority in ranking; simultaneously, to enhance the system's ability to respond quickly to sudden events, the timeliness weight of these information units can also be multiplied by a coefficient greater than 1, such as 1.5.

[0130] Conversely, if the system detects that certain information units are outdated or less relevant to the current business context, it can also reduce the weight of these information units. For example, if a news report about a company's historical performance is no longer relevant in the current market environment, the system can multiply the timeliness score of that information unit by a coefficient less than 1, such as 0.8, to lower its priority in the ranking. Adjustments to timeliness weights can be made in a similar manner.

[0131] Optionally, the adaptive weight adjustment mechanism can employ a rule-based strategy. For example, a series of rules can be pre-defined, triggering corresponding weight adjustments when specific conditions are met. For instance, the rules could be set as follows: when a company issues a profit warning, the timeliness score of all financial analysis reports related to that company is increased by 20%; or, when a major policy adjustment occurs in an industry, the timeliness weight of all research reports related to that industry is increased by 30%. In other optional implementations, the adaptive weight adjustment mechanism can employ a machine learning-based strategy. For example, a weight adjustment model can be trained using historical data, which can automatically predict the weight adjustment value for each information unit based on current business scenario data. This model can use deep learning algorithms, such as recurrent neural networks or Transformer models, to learn the complex relationship between business scenario data and weight adjustment values.

[0132] The system will dynamically adjust the timeliness and timeliness weights of information units based on the results of the event detection model or preset rules, and sort the information units according to the adjusted weights. This will enable the system to respond more flexibly to complex business scenarios and generate more accurate and comprehensive decision-making information.

[0133] In this way, the system can capture changes in business scenarios in real time and quickly adjust information processing strategies, thereby ensuring that decision-makers can obtain the most relevant and accurate information in a timely manner, improving decision-making efficiency and quality.

[0134] In some embodiments, for the semantic and logical connections between information units mined through association analysis algorithms, the system constructs an association information network data structure in memory to represent the relationships between information units. This data structure can be implemented using a graph database or an in-memory graph data structure. Specifically, for semantic connections, as in step S200 of the above embodiments, the Sentence-BERT model can be used to convert the text content of each information unit into a 768-dimensional semantic vector. The cosine similarity between any two information unit vectors is calculated. If the similarity is greater than 0.8, they are considered to have a strong semantic connection. For logical connections, topic modeling (LDA) can be used to mine topic connections between information units. For example, the text set of all selected information units is input into an LDA topic model, which is preset to extract 20 topics. Information units belonging to the same topic with the highest probability are considered to have a macro-topic connection. The data structure of the association information network is represented by a graph data structure, where each information unit corresponds to a node in the graph, and the semantic and logical connections between information units correspond to edges in the graph. The weights of the edges can be set to the corresponding similarity or connection strength. For example, semantic similarity of 0.8 can be set as the edge weight, while topic association can be set to a lower weight, such as 0.3, to distinguish the strength of different types of associations. This graph data structure can be stored in computer memory, such as in memory or a database. Graph databases such as Neo4j can provide efficient graph data storage and retrieval capabilities. In-memory graph data structures can be implemented using Python's NetworkX library, providing fast graph algorithm operations.

[0135] The association information network consists of nodes and edges. Nodes represent information units, storing attributes such as text content, gender rating, and timeliness weight. Edges represent the relationships between information units, storing attributes such as the type of relationship (e.g., semantic or logical) and the strength of the relationship (e.g., weight). The association information network is used to store and query the relationships between information units. By querying the association information network, other information units related to a given information unit can be quickly found, thus enabling information association analysis and reasoning. When a new information unit is added to the system, the system first calculates the semantic similarity and topic relevance between that information unit and other information units. Then, based on the calculation results, corresponding nodes and edges are added to the association information network. When it is necessary to query information related to a given information unit, the system first finds the node corresponding to that information unit in the association information network, and then uses graph algorithms (e.g., depth-first search or breadth-first search) to find other nodes connected to that node. These nodes correspond to the information units related to that given information unit.

[0136] Optionally, the semantic vector dimension is not limited to 768 dimensions. Different dimensions can be selected according to the actual situation, such as 512 dimensions or 1024 dimensions.

[0137] Optionally, the number of theme models is not limited to 20; different numbers of themes can be selected according to the actual situation.

[0138] Alternatively, the associated information network can be implemented using other graph data structures, such as adjacency matrices or adjacency lists.

[0139] By constructing a relational information network data structure, the system can effectively store and query the relationships between information units, thereby enabling relational analysis and reasoning. This helps the system better understand the relationships between information units and generate more comprehensive and accurate decision-making information. For example, when the system needs to assess the risk of a company, it can query the relational information network to find all information units related to that company, thus comprehensively assessing the company's risk status and solving the problem of information omissions caused by insufficient information relational mining in existing technologies.

[0140] like Figure 2 As shown, this embodiment of the invention provides a decision information generation system based on multi-objective reinforcement learning. The system consists of an information processing module M100, a subset generation module M200, a sorting module M300, a decision generation module M400, and a feedback adjustment module M500. The modules work together to realize the decision information generation method based on multi-objective reinforcement learning in any of the above embodiments, thereby realizing the generation and optimization of decision information.

[0141] The information processing module M100 is configured to acquire multi-dimensional information flow data containing timestamps and business tags, and to process information units within the information flow data using a classification model to determine their timeliness score and weight. One implementation involves the system capturing financial news, social media posts, and internal business system data in real time from multiple pre-defined application programming interfaces (APIs). For example, each financial news item retrieved from a news aggregation platform includes a title, body text, and publication timestamp; each post retrieved from a social media platform includes text content and a timestamp; and transaction data retrieved from the internal business system database includes stock codes, alert times, and alert types. For each piece of unstructured text data acquired, the information processing module M100 inputs it as an information unit into a pre-trained classification model. This classification model can be based on the BERT architecture and fine-tuned using financial domain corpora. The classification model outputs a business tag for the information unit, such as "macroeconomics," "company financial reports," "regulatory policies," or "market rumors." Meanwhile, the model outputs a floating-point number between 0 and 1 as the initial score of the information unit, and a timeliness weight calculated by an exponential decay function based on the difference between the timestamp of the information unit and the current system time.

[0142] Alternatively, the classification model can also use other natural language processing models, such as RoBERTa or XLNet.

[0143] The subset generation module M200 is configured to process information units with a targeting score higher than a preset first threshold and a timeliness weight that meets the requirements of a preset business scenario. It employs an association analysis algorithm and a clustering algorithm to generate information subsets that are associated with a preset business element and meet a preset coverage. One implementation involves first selecting information units with a targeting score higher than 0.7 and a timeliness weight higher than 0.5. Then, the Sentence-BERT model is used to convert the text content of each selected information unit into a 768-dimensional semantic vector. The cosine similarity between any two information unit vectors is calculated. If the similarity is greater than 0.8, a strong semantic association is considered to exist between them, and an edge with the cosine similarity weight is added to these two nodes in a graph data structure constructed in memory. Simultaneously, the text set of all selected information units is input into an LDA topic model, which is preset to extract 20 topics. For information units belonging to the same topic with the highest probability, a lower-weighted edge is added to them in the association information network, indicating that they have a macro-topic association. The Louvain community detection algorithm is used to process the associated information network, dividing information units into different groups. The system pre-defines business elements, such as "XX company's debt default risk," and represents it as a 768-dimensional target semantic vector. The system calculates the average semantic vector of all information units in each group and identifies the group with the highest cosine similarity to the target semantic vector, using it as the initial information subset. If the coverage of this information subset with the business elements—that is, the cosine similarity between the subset's average vector and the target semantic vector—is less than 0.85, the system calculates the difference vector between the subset's average vector and the target semantic vector, representing the semantic gap. The system analyzes the semantic concept corresponding to the dimension with the largest value in this difference vector, such as the concept related to "legal proceedings." The system automatically combines this semantic concept with the business elements to form a new query term, such as "XX company's legal proceedings," initiates a supplementary query to an external search engine API, adds the new information units found in the query to the information subset, recalculates the coverage, and repeats this process until the requirements are met.

[0144] Optionally, the association analysis algorithm can use knowledge graph methods to associate information units with a pre-built financial knowledge graph, thereby uncovering deeper relationships.

[0145] The sorting module M300 is configured to employ an adaptive weight adjustment mechanism, updating the attribute weights and timeliness weights of each information unit in the information subset based on real-time changes in the business scenario, and then sorting the information units. One implementation involves a tactical agent in the system monitoring external "stock price fluctuation" data streams in real time. When it detects that the stock price of "Company XX" has fallen by more than 5% within 5 minutes, the tactical agent immediately executes an action, temporarily increasing the attribute score weights of all information units related to "Company XX" by 1.5 times and the timeliness weights by 2 times. Then, for each unit in the information subset, its comprehensive score is calculated using the formula:

[0146]

[0147] in and These are the basic weights, which can be adjusted by the strategic agent. The actions of the tactical agent actually temporarily amplify these two weights. According to... Sort the information units in descending order. Traverse the sorted list, and if the cosine similarity of the semantic vectors of two adjacent information units is higher than 0.95, keep the one with the higher score and discard the one with the lower score.

[0148] Optionally, the adaptive weight adjustment mechanism can be implemented using a rule engine, which pre-sets a series of rules and triggers the corresponding weight adjustment operation when specific conditions are met.

[0149] The decision generation module M400 is configured to generate structured decision information based on the sorting results of the sorting module M300 using a preset decision template. One implementation involves the system filling the final sorted and deduplicated information list with a preset JSON decision template to generate structured decision information. For example, the generated JSON data includes fields such as risk subject, risk level, information source, score, content, and generation time. Optionally, the decision template can use other formats such as XML or YAML.

[0150] The feedback adjustment module of M500 is configured to adjust the parameters in the classification model and adaptive weight adjustment mechanism through a feedback mechanism based on the deviation between structured decision information and an actual business scenario. One implementation involves assuming that XX company actually issues a debt default announcement 24 hours later. The system compares this actual business outcome with the previously generated high-risk decision information and finds that the deviation is very small. The system calculates the reward value based on a preset reward function, the formula of which is...

[0151]

[0152] in For deviation, This is a quantitative measure of the positive business impact of this successful early warning. This represents the weighting coefficient. This positive reward is fed back to the tactical and strategic agents. The tactical agent updates its policy network using algorithms such as Q-learning, making it more inclined to make similar weight-increasing actions when "stock price fluctuations" occur. The strategic agent, based on the cumulative reward over a longer period, decides whether to perform strategic actions such as adjusting the hyperparameters of the underlying classification model or the number of LDA topics, aiming to fundamentally improve system performance.

[0153] Optionally, the feedback adjustment module that makes up the M500 can employ other reinforcement learning algorithms, such as the Actor-Critic algorithm or the Trust Region Policy Optimization (TRPO) algorithm.

[0154] Through the collaborative work of the above modules, this decision information generation system can achieve dynamic balance across multiple objectives and possesses self-learning and evolutionary capabilities. The information processing module M100 is responsible for initial information screening and processing; the subset generation module M200 is responsible for mining the relationships between information and generating information subsets related to business elements; the sorting module M300 is responsible for adjusting information weights and sorting the information according to real-time changes in the business scenario; the decision generation module M400 is responsible for transforming the sorted information into structured decision information; and the feedback adjustment module M500 is responsible for adjusting the parameters in the classification model and adaptive weight adjustment mechanism based on the application effect of the decision information, thereby improving the overall performance of the system.

[0155] In some embodiments, the feedback adjustment module in the system constitutes M500, which is further configured to execute a feedback mechanism through a cooperative multi-agent reinforcement learning architecture to achieve dynamic optimization of the information processing module M100, the subset generation module M200, and the sorting module M300.

[0156] Specifically, the collaborative multi-agent reinforcement learning architecture is implemented in this system as follows:

[0157] 1. Architecture Composition: The architecture consists of two agents: a strategic agent and a tactical agent. Both are deep neural networks and have independent storage space allocated in the computer memory to store their respective network parameters.

[0158] 2. Strategic Agent: The strategic agent is primarily responsible for learning and optimizing long-term strategies. Its input is data on the application effects of historical decision-making information. For example, it might compare all decision-making information generated in the past month with the actual business results, calculate metrics such as accuracy and recall for each decision, and compile these metrics into a comprehensive evaluation report. Based on this report, the strategic agent decides whether to adjust the parameters of the underlying model or algorithm. For example, if it finds that the prediction accuracy of recently generated decision-making information for a specific type of risk is consistently low, the strategic agent might decide to increase the weight of that type of risk in the classification model or adjust the number of topic models in the association analysis algorithm. The strategic agent is trained using deep reinforcement learning algorithms, such as proximal policy optimization algorithms, to learn how to make optimal parameter adjustment decisions in different business scenarios. The output of the strategic agent is specific parameter adjustment instructions, such as "adjust the learning rate of the BERT classification model to 0.0001" or "increase the number of topics in the LDA topic model to 30," etc. These instructions are passed to the information processing module M100 and the subset generation module M200 to update their internal model parameters.

[0159] 3. Tactical Agent: The tactical agent is primarily responsible for learning and optimizing short-term strategies. Its input is real-time data streams from the business scenario. For example, this could be real-time data such as stock prices and trading volumes obtained from a stock trading system, or public opinion information related to specific companies scraped from social media. Based on this real-time data, the tactical agent determines whether the current market environment has changed significantly and decides whether to adjust the gender and timeliness weights of information units. For example, if abnormal fluctuations in a company's stock price are detected, the tactical agent might decide to temporarily increase the gender weight of information units related to that company so that these information units are prioritized in the ranking module M300. The tactical agent is also trained using deep reinforcement learning algorithms, such as deep Q-networks, to learn how to make optimal weight adjustment decisions in different real-time scenarios. The output of the tactical agent is specific weight adjustment instructions, such as "increase the gender weight of all information units related to XX company by 2 times" or "increase the timeliness weight of all information units published within 1 hour by 1.5 times." These instructions are passed to the ranking module M300 to update the overall score of the information units.

[0160] 4. Information Transfer and Collaboration: The strategic and tactical agents exchange information through a shared knowledge base. This knowledge base stores the learning experiences and strategies of each agent and is periodically updated. For example, the tactical agent can share recently successful weight adjustment strategies with the strategic agent for reference and learning; the strategic agent can also inform the tactical agent of the optimization direction of the underlying model, helping it to better understand changes in the business scenario.

[0161] 5. Reward Function Design: To guide the two agents towards a common goal, the system employs a global reward function. This function comprehensively considers multiple metrics such as the accuracy, recall, and coverage of decision information, and is adjusted according to the needs of specific business scenarios. For example, in a financial risk monitoring scenario, the reward function may place greater emphasis on the accuracy of decision information to avoid losses due to false alarms; while in a public opinion analysis scenario, the reward function may place greater emphasis on the recall of decision information to avoid missing information. The reward function feeds back the reward value to the two agents to update their policy networks.

[0162] Alternatively, strategic and tactical agents can employ other deep learning models, such as Transformer networks and recurrent neural networks. Furthermore, other mechanisms can be used for information exchange between the two agents, such as attention mechanisms and message passing networks.

[0163] Optionally, the architecture can be extended to a collaborative multi-agent system with more agents, such as adding a "data agent" specifically responsible for acquiring high-quality data from external data sources and preprocessing the data to improve the overall performance of the system.

[0164] By employing the aforementioned collaborative multi-agent reinforcement learning architecture, the feedback adjustment module M500 can dynamically adjust the parameters of the information processing module M100, subset generation module M200, and sorting module M300 based on feedback from actual business scenarios, thereby achieving end-to-end optimization of the decision information generation process. This self-learning and evolutionary capability enables the system to continuously adapt to complex and ever-changing environments and continuously improve the quality and efficiency of decision information.

[0165] In some embodiments, the collaborative multi-agent reinforcement learning architecture includes a strategic agent and a tactical agent, which work together to achieve adaptive optimization of the decision information generation system. The strategic agent is configured to perform a first action—adjusting the hyperparameters of a classification model or association analysis algorithm—based on a first state determined by the long-term application effects of the decision information. For example, the strategic agent can monitor the accuracy of risk reports generated by the system over the past week, and the impact of these reports on actual business operations, such as the amount of losses avoided. If the accuracy consistently falls below a preset threshold (e.g., 80%), or the amount of losses avoided is lower than expected, the strategic agent determines that the system is in a state requiring adjustment. At this point, the strategic agent can perform actions, such as adjusting the learning rate of the BERT classification model or adjusting the number of topics extracted by the LDA topic model.

[0166] The specific implementation of adjusting the learning rate of the BERT classification model is as follows: the strategic agent sends an instruction to the information processing module M100, which contains a new learning rate value. After receiving the instruction, the information processing module M100 updates the learning rate parameter in the optimizer of the BERT classification model.

[0167] The specific implementation method for adjusting the number of topics extracted by the LDA topic model is as follows: the strategic agent sends an instruction to the subset generation module M200, which contains a new number of topics. After receiving the instruction, the subset generation module M200 reinitializes the LDA topic model and trains it using the new number of topics.

[0168] Optionally, the strategic agent can also adjust the weights of topic association and semantic association in the hybrid association analysis algorithm. For the tactical agent, it is configured to perform a second action—adjusting the sex weights and timeliness weights used for ranking—based on a second state constituted by real-time changes in the business scenario. For example, the tactical agent can monitor real-time stock price fluctuations. If it detects a significant drop in a stock price within a short period, such as a drop of more than 5% within 5 minutes, the tactical agent determines that a sudden event has occurred in the current business scenario. At this time, the tactical agent can perform actions, such as increasing the sex weights and timeliness weights of risk information related to that stock.

[0169] The specific implementation of the enhancement weight is as follows: The tactical agent sends an instruction to the sorting module M300, which includes the identifier of the information unit to be adjusted and the adjustment range. After receiving the instruction, the sorting module M300 adjusts the relevant information unit according to the instruction. In the calculation formula Parameter value.

[0170] The specific implementation method for enhancing the timeliness weight is as follows: The tactical agent sends an instruction to the sorting module M300, which includes the identifier of the information unit to be adjusted and the adjustment range. After receiving the instruction, the sorting module M300 adjusts the relevant information unit according to the instruction. In the calculation formula Parameter values. In some alternative implementations, the tactical agent can monitor changes in public opinion about a specific company on social media, such as a sharp increase in the number of negative comments, and adjust relevant weights based on this information.

[0171] In the above implementation, the strategic agent is responsible for long-term, global optimization, while the tactical agent is responsible for short-term, local adjustments. The two agents collaborate to drive the system to continuously learn and evolve in a dynamic environment, thereby improving the quality and efficiency of decision-making information generation. The strategic agent adjusts the underlying model parameters to make the system more adaptable to changes in the business environment. The tactical agent adjusts ranking weights to enable the system to respond more quickly to unexpected events and capture information.

[0172] In some embodiments, the subset generation module M200 in this system performs clustering and subset extraction as follows:

[0173] 1. Clustering Algorithm Implementation: The subset generation module M200 calls the Louvain community discovery algorithm program, which is a graph clustering algorithm based on modularity optimization. This algorithm takes a network of associated information (stored in memory) as input, where nodes represent information units, and edges represent the strength of association between information units (semantic similarity or topic relevance). The Louvain algorithm first treats each node as an independent community, then iteratively moves nodes to neighboring communities to maximize the modularity of the entire network. Modularity is a metric for measuring the quality of community partitioning; a higher value indicates a more pronounced community structure. The algorithm repeats this process until the modularity no longer significantly improves. For example, the Louvain algorithm can divide financial news, social media posts, and insider trading data into multiple groups, where information units within each group have high correlations, while the correlations between different groups are low.

[0174] Alternatively, other graph clustering algorithms can be used, such as spectral clustering algorithms or label propagation-based algorithms.

[0175] 2. Grouping Extraction: In this embodiment, the business element is pre-represented as a 768-dimensional target semantic vector, which is stored in memory. The subset generation module M200 calculates the average semantic vector of all information units in each group. Specifically, this is done by adding the 768-dimensional semantic vectors of all information units in the group dimension by dimension, and then dividing by the number of information units in the group. The semantic vector of each information unit is generated by the Sentence-BERT model and stored in memory. Then, the subset generation module M200 calculates the cosine similarity between the average semantic vector of each group and the target semantic vector. The higher the cosine similarity, the higher the matching degree between the group and the business element. The subset generation module M200 selects the group with the highest cosine similarity as the initial information set. For example, if the business element is "XX Company Credit Risk", the Louvain algorithm may generate multiple groups, one of which contains financial news, analyst reports, and social media discussions related to XX Company. The subset generation module M200 calculates the cosine similarity between each group and the target semantic vector of "XX Company Credit Risk" and selects the group with the highest similarity as the initial information set.

[0176] Alternatively, other vector similarity metrics can be used, such as Euclidean distance or Manhattan distance.

[0177] 3. Coverage Calculation: The subset generation module M200 calculates the coverage of the current information set to the business elements. In this embodiment, coverage is defined as the cosine similarity between the average semantic vector of the information set and the target semantic vector. If the cosine similarity is lower than a preset coverage threshold (e.g., 0.85), the subset generation module M200 triggers an iterative query mechanism to supplement the missing information.

[0178] 4. Iterative Query Mechanism: The subset generation module M200 analyzes the semantic gap between the average semantic vector of the information set and the target semantic vector. The semantic gap is defined as the difference vector obtained by subtracting the average semantic vector of the information set from the target semantic vector. The subset generation module M200 analyzes the semantic concept corresponding to the dimension with the largest value in this difference vector. For example, if the dimension related to "legal proceedings" has the largest value in the difference vector, it indicates that the current information set lacks information related to legal proceedings of XX Company. Then, the subset generation module M200 automatically combines this semantic concept with business elements to form a new query term, such as "legal proceedings of XX Company," and initiates a supplementary query to an external search engine API. The external search engine API can be pre-defined, such as the Baidu search engine API or the Google search engine API. The subset generation module M200 parses the results returned by the search engine, extracts new information units, and adds these information units to the information set. The subset generation module M200 recalculates the average semantic vector and coverage of the information set and repeats this process until the coverage meets the requirements.

[0179] Alternatively, other semantic gap analysis methods can be used, such as knowledge graph-based methods.

[0180] Alternatively, other query methods can be used, such as using a pre-built knowledge base instead of a search engine.

[0181] 5. Information Subset Generation: After the above steps of clustering, subset extraction, coverage calculation, and iterative query, the subset generation module M200 finally generates an information subset. This subset is associated with preset business elements and meets preset coverage requirements. This information subset is then passed to the sorting module M300 for further processing.

[0182] Through the above methods, the subset generation module M200 can effectively extract high-quality information subsets related to business elements, providing a reliable information foundation for subsequent decision-making information generation. This module can supplement missing information through an iterative query mechanism, improving information coverage and thus reducing decision-making biases caused by information omissions.

[0183] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for generating decision information based on multi-objective reinforcement learning, wherein the method is executed by a computer device, characterized in that, include: Acquire multidimensional information flow data containing timestamps and business tags, and use a classification model to process the information units in the information flow data to determine their importance score and timeliness weight; For information units whose importance score is higher than the preset first threshold and whose timeliness weight meets the preset business scenario requirements, association analysis algorithm and clustering algorithm are used to process them in order to generate information subsets that are associated with preset key business elements and meet preset coverage. An adaptive weight adjustment mechanism is adopted to update the importance weight and timeliness weight of each information unit in the information subset according to the real-time changes of the business scenario, and to sort the information units. Based on the sorting results, structured decision information is generated using a preset decision template; Based on the deviation between the structured decision information and the actual business scenario, the parameters in the classification model and the adaptive weight adjustment mechanism are adjusted through a feedback mechanism. The feedback mechanism is executed by a collaborative multi-agent reinforcement learning architecture, including: a strategic agent configured to: perform a first action of adjusting the hyperparameters of the classification model or the association analysis algorithm based on a first state constituted by the long-term application effect of the decision information; and a tactical agent configured to: perform a second action of adjusting the importance weights and timeliness weights used for ranking based on a second state constituted by the real-time changes in the business scenario. Based on the deviation and / or the long-term application effect of the decision information, a reward value is calculated, and the reward value is used to update the strategies of the strategic agent and the tactical agent respectively.

2. The method according to claim 1, characterized in that, The association analysis algorithm is a hybrid association analysis algorithm, which includes: The topic model is used to mine the topic associations between the information units; A word embedding model is used to calculate the semantic similarity between the information units in order to mine the semantic associations between the information units.

3. The method according to claim 1, characterized in that, The step of generating an information subset that satisfies a preset coverage includes: The clustering algorithm is used to group the information units, and the information set with the highest matching degree with the key business element is extracted from each group. In response to the information set having a lower coverage of the key business elements than the preset coverage, the missing information is supplemented through an iterative query mechanism to generate the information subset.

4. The method according to claim 3, characterized in that, Before the iterative query mechanism supplements the missing information, the method further includes: The key business elements are represented as target semantic vectors; The semantic gap between the two is analyzed by calculating the vector distance between the vector representation of the information set and the target semantic vector. The iterative query mechanism generates supplementary query instructions based on the semantic gap to obtain supplementary information.

5. The method according to claim 1, characterized in that, The adaptive weight adjustment mechanism is triggered based on real-time changes in the business scenario to increase or decrease the importance weight and / or timeliness weight of specific information units related to the real-time changes.

6. The method according to claim 1, characterized in that, The method further includes: In the memory, the semantic and logical relationships between the information units mined by the association analysis algorithm are constructed into an associated information network data structure.

7. A decision information generation system based on multi-objective reinforcement learning, characterized in that, include: The information processing module is configured to acquire multidimensional information flow data containing timestamps and business tags, and to process the information units in the information flow data using a classification model to determine their importance score and timeliness weight. The subset generation module is configured to process information units with importance scores higher than a preset first threshold and timeliness weights that meet the requirements of preset business scenarios using association analysis algorithms and clustering algorithms to generate information subsets that are associated with preset key business elements and meet preset coverage. The sorting module is configured to use an adaptive weight adjustment mechanism to update the importance weight and timeliness weight of each information unit in the information subset according to the real-time changes of the business scenario, and to sort the information units. The decision generation module is configured to generate structured decision information based on the sorting results of the sorting module using a preset decision template. The feedback adjustment module is configured to adjust the parameters in the classification model and the adaptive weight adjustment mechanism based on the deviation between the structured decision information and the actual business scenario through a feedback mechanism. The feedback mechanism is executed by a collaborative multi-agent reinforcement learning architecture, comprising: a strategic agent configured to: perform a first action of adjusting the hyperparameters of the classification model or the association analysis algorithm based on a first state constituted by the long-term application effect of the decision information; and a tactical agent configured to: perform a second action of adjusting the importance weights and timeliness weights used for ranking based on a second state constituted by real-time changes in the business scenario; calculate reward values ​​based on the deviation and / or the long-term application effect of the decision information, and update the strategies of the strategic agent and the tactical agent using the reward values ​​respectively.

Images