Big data intelligent decision analysis method and system based on machine learning

Abstract

The invention provides a big data intelligent decision analysis method and system based on machine learning. The method comprises the following steps: receiving real-time data streams of a plurality of sources, the real-time data streams comprising structured data and unstructured data; analyzing the unstructured data by applying semantic understanding and context sensing technologies so as to extract themes and emotional tendencies in the unstructured data; the emphasis of data analysis is dynamically adjusted, and an association network between data is constructed by using a graph technology; evaluating mutual influence among different data entities, and predicting future state change to obtain a prediction result; based on the mode, the trend and the prediction result, generating a strategy suggestion oriented to a specific application scene; and the strategy suggestions are converted into executable service rules to support intelligent decision making. According to the technical scheme provided by the invention, different types of data can be comprehensively analyzed to identify key trends, understand emotions of consumers and predict market trends, so that the data analysis efficiency is effectively improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data analysis technology, and in particular to a big data intelligent decision analysis method and system based on machine learning. Background Art

[0002] In a rapidly changing business environment, enterprises and organizations need to be able to make decisions quickly to respond to market dynamics. This requires them to not only process large amounts of structured and unstructured data in real time, but also to extract valuable information from this data.

[0003] There are many data analysis solutions on the market that aim to help organizations better understand and utilize big data. One of the more advanced methods is to use a big data intelligent decision analysis system based on machine learning. The system first integrates real-time data streams from different channels (such as social media platforms, online forums, etc.); then, it uses natural language processing technology to parse unstructured text content and extract topic and sentiment information from it; then, it uses graph technology to build a knowledge graph that represents the relationship between data; finally, it uses complex machine learning algorithms to discover patterns and trends hidden in the association network and make strategic recommendations based on this.

[0004] Although existing machine learning-based big data intelligent decision analysis solutions have shown great potential, the understanding of unstructured data relies on high-quality language models, but current technologies may not be able to accurately capture certain nuances or culturally specific expressions, resulting in deviations in sentiment analysis or semantic understanding. Summary of the invention

[0005] The embodiments of the present application provide a big data intelligent decision analysis method and system based on machine learning to solve the problem of poor traditional data analysis in the prior art.

[0006] In a first aspect, the present application provides a big data intelligent decision analysis method based on machine learning, including: Receiving real-time data streams from multiple sources, the real-time data streams including structured data and unstructured data; Applying semantic understanding and context awareness technology to parse the unstructured data to extract topics and sentiment tendencies in the unstructured data; Dynamically adjust the focus of data analysis based on the topic and sentiment, and use graph technology to build a network of associations between data; Analyze patterns and trends in the association network through machine learning models, evaluate the mutual influence between different data entities, and predict future state changes to obtain prediction results; Based on the pattern, the trend and the prediction result, generate strategy recommendations for specific application scenarios; Translate the policy recommendations into executable business rules to support intelligent decision making.

[0007] Optionally, dynamically adjusting the focus of data analysis according to the topic and sentiment tendency, and using graph technology to construct an association network between data, includes: Using a dynamic weight allocation mechanism, different data types included in the topic and sentiment tendency are evaluated for importance scoring, and a dynamic weight allocation strategy is obtained; According to the dynamic weight allocation strategy, the processing priorities of different types of data are adjusted in real time to obtain adjusted data processing priorities; Based on the adjusted data processing priority, the processing depth of the corresponding real-time data stream is enhanced or weakened to dynamically adjust the focus of data analysis; Performing entity recognition processing on the unstructured data to obtain entities and their relationships; According to the entities and their relationships, a graph construction algorithm is used to perform graph conversion processing on the entities and their relationships to generate a preliminary association network; The preliminary association network is processed for pattern and trend discovery to identify indirect associations between nodes, and the preliminary association network is processed for expansion to generate an association network by introducing rules provided by an external knowledge base or domain experts.

[0008] Optionally, analyzing patterns and trends in the association network through a machine learning model, evaluating the mutual influence between different data entities, and predicting future state changes to obtain prediction results includes: Using the association network, link processing is performed on the relationship between the topics and sentiment tendencies and the node-based key information in the structured data to obtain node-based relationship links; Analyzing and processing the patterns and trends in the association network according to the node-based relationship links to obtain the trends of changes of different data entities in the association network over time; Based on the node-based relationship links, the mutual influence between the data entities in the association network is evaluated and processed, and the machine learning model is trained based on the trends of different data entities changing over time and the evaluation results; By using the trained machine learning model, the future state is predicted based on the state of the associated network and the node-based relationship links in the current and past period of time, and the predicted value of the future state change is generated to obtain the prediction result.

[0009] Optionally, a dynamic weight allocation mechanism is used to evaluate the importance of different data types contained in the topic and sentiment tendency, including: The importance score of a data type is calculated using the following formula : ; in, It is The importance score of each data source, is the number of topics or sentiment dimensions considered, It is The time-dependent weight coefficient of each topic score, It is The data source is in The scores on the subject, It is The quadratic coefficient of the subject score is It is The time-dependent weight coefficient of the sentiment tendency score, It is The data source is in The scores on the emotional dimension, It is The absolute value coefficient of the sentiment score, is the time-dependent weight coefficient of the data source reliability score, It is The reliability score of each data source, is the distance factor, It is The Euclidean distance between each data source and the central data point, is the time sensitivity factor, It is Time correlation indicators for data sources, is the position sensitivity factor, It is Location relevance indicators for data sources, is the context sensitivity factor, It is Contextual consistency scores for data sources.

[0010] Optionally, the method uses the trained machine learning model to perform prediction processing of future states according to the states of the associated networks in the current and past period of time and the node-based relationship links, generates prediction values ​​about future state changes, and obtains prediction results, including: The following formula is used to combine the machine learning model to generate predictions about future state changes and obtain the prediction results : ; in, is an evaluation indicator of the overall prediction performance of the model. is the number of test samples, is the time point or sample number, It's time point The absolute error between the predicted result of the above model and the true value, is the accuracy of causal inference, at the time point The degree to which the above model can correctly identify causal relationships, is variability, It's time point On The actual value of the external factor, It's time point On The predicted value of an external factor, is the number of external factors considered, It is an external factor The influence weight of is the graph similarity index, They are the absolute error over time, the accuracy of causal inference, the influence of variability and external factors, and the weight coefficient of graph similarity, which also need to satisfy conditions, is the decay rate of absolute error, It is variability The standard deviation of .

[0011] Optionally, applying semantic understanding and context awareness technology to parse the unstructured data to extract topics and sentiment tendencies in the unstructured data includes: Process the received unstructured text data in real time to obtain processed text data; Performing vectorization processing on the words in the processed text data to obtain a semantic relationship representation between the words; Performing entity information recognition processing on the processed text data to obtain entity information; Performing sentence structure analysis on the processed text data to obtain grammatical dependency relationships between words; Based on the entity information and the grammatical dependency relationship between words, subject extraction processing is performed on the processed text data to generate the main discussion subject or topic; The processed text data is subjected to context information processing to obtain context information at the sentence and paragraph levels, and the processed text data is subjected to sentiment tendency assessment processing based on the context information at the sentence and paragraph levels to obtain a recognition result of the sentiment tendency.

[0012] Optionally, the generating a strategy suggestion for a specific application scenario based on the pattern, the trend and the prediction result includes: Extract key information points from the time-varying trends of different data entities in the association network to obtain potential change rules; Analyze the prediction results, identify changes of future events on the prediction results, and obtain the impact of the changes on the current decision-making environment; According to the characteristics of the application scenario, define the objective function, quantify the effectiveness and feasibility of the strategy recommendations, and obtain quantitative indicators; Combining the potential change rules, the impact of the change on the current decision-making environment and the quantitative indicators, a multi-criteria decision analysis method is applied to generate and process the strategic alternatives to obtain a set of strategic alternatives; The performance of the strategy alternatives set in different scenarios is evaluated and processed to screen out the optimal strategy recommendation.

[0013] In a second aspect, the present application embodiment provides a big data intelligent decision analysis system based on machine learning, comprising: A receiving module, used for receiving real-time data streams from multiple sources, wherein the real-time data streams include structured data and unstructured data; An extraction module, used for applying semantic understanding and context awareness technology to parse the unstructured data to extract themes and sentiment tendencies in the unstructured data; An adjustment module, used to dynamically adjust the focus of data analysis according to the topic and sentiment tendency, and to construct an association network between data using graph technology; An evaluation module, used to analyze patterns and trends in the association network through a machine learning model, evaluate the mutual influence between different data entities, and predict future state changes to obtain prediction results; A generation module, used to generate a strategy suggestion for a specific application scenario based on the pattern, the trend and the prediction result; The conversion module is used to convert the policy recommendations into executable business rules to support intelligent decision making.

[0014] In a third aspect, an embodiment of the present application provides a computing device, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a big data intelligent decision analysis method based on machine learning as described in any one of the first aspects above.

[0015] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements a big data intelligent decision analysis method based on machine learning as described in any one of the first aspects above.

[0016] In the embodiment of the present application, real-time data streams from multiple sources are received, and the real-time data streams include structured data and unstructured data; semantic understanding and context-aware technology are applied to parse unstructured data to extract themes and emotional tendencies in unstructured data; the focus of data analysis is dynamically adjusted, and the association network between data is constructed using graph technology; the mutual influence between different data entities is evaluated, and future state changes are predicted to obtain prediction results; based on patterns, trends and prediction results, strategy recommendations for specific application scenarios are generated; and the strategy recommendations are converted into executable business rules to support intelligent decision-making. The present application can process real-time data streams from multiple sources, not only limited to structured data, but also unstructured data, thereby improving the comprehensiveness and flexibility of data processing. By applying semantic understanding and context-aware technology, valuable themes and emotional tendencies can be effectively extracted from unstructured data, providing a basis for subsequent data analysis. Using graph technology to construct an association network between data helps to reveal the complex relationships between different data entities and provide support for in-depth analysis. By analyzing the patterns and trends in the association network, future state changes can be accurately predicted, and specific strategy recommendations can be generated accordingly to help decision makers make more informed choices. Converting strategic recommendations into executable business rules directly supports intelligent decision-making and improves the efficiency and quality of decision-making.

[0017] Furthermore, through the dynamic weight allocation mechanism, the priority of data processing can be flexibly adjusted according to the importance of the subject and sentiment of the data, ensuring the pertinence and effectiveness of data analysis. According to the dynamic weight allocation strategy, the depth of data processing is adjusted in real time, which not only ensures the in-depth mining of key information, but also avoids the waste of resources. Entity recognition is performed on unstructured data, and the key entities and their relationships in the data are accurately obtained, laying the foundation for building a high-quality association network. Combining entities and their relationships, the graph construction algorithm is used to generate a preliminary association network, and it is optimized through pattern and trend discovery processing, which improves the quality and practicality of the association network.

[0018] Furthermore, by node-based processing of key information in topics, sentiment tendencies, and structured data and establishing relationship links, we have deepened our understanding of the intrinsic connections between data. Based on node-based relationship links, we can more accurately analyze the changing trends of different data entities in the association network over time, providing a solid foundation for prediction. By evaluating the mutual influence between data entities in the association network, we ensure the scientificity and rationality of model training and improve the model's predictive ability. Using the trained machine learning model, we can effectively predict future development trends based on the current and past data status, providing an important reference for decision-making. By generating predicted values ​​for future state changes, we can provide decision makers with forward-looking guidance and enhance support for decision-making.

[0019] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flowchart of a big data intelligent decision analysis method based on machine learning provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a big data intelligent decision analysis system based on machine learning provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0023] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0025] Figure 1 A flowchart of a big data intelligent decision analysis method based on machine learning is provided for the embodiment of the present application, such as Figure 1 As shown, the method includes: 101. Receive real-time data streams from multiple sources, where the real-time data streams include structured data and unstructured data; In this step, the real-time data stream includes but is not limited to data obtained from sources such as sensors, monitoring equipment, user terminals, social media platforms, etc. In the field of energy and power, these data can be divided into structured data and unstructured data.

[0026] Among them, structured data usually comes from SCADA (Supervisory Control and Data Acquisition) systems, meter readings, power grid operation data, etc. This type of data has a fixed format and is easy to store and process. For example, monitoring data such as voltage, current, frequency, and temperature in the power grid.

[0027] Among them, unstructured data includes social media comments, news reports, user feedback, etc. This type of data is diverse in form and requires additional technical means to process before it can be used. For example, users' evaluation of power services, on-site picture descriptions of power failures, etc.

[0028] In the embodiment of the present application, for example, the voltage and current data of each substation can be collected once a minute through the power grid monitoring system; or the smart meter can upload the user's electricity consumption every hour, etc., to obtain structured data. Unstructured data can be obtained through comments about power services posted by users on social media platforms; or text descriptions and pictures contained in power failure reports, etc. 102. Applying semantic understanding and context awareness technology to parse the unstructured data to extract themes and sentiment tendencies in the unstructured data; In this step, semantic understanding technology can identify key information in unstructured text, such as events, places, times, etc. Context-aware technology considers the context in which the data is generated to improve the accuracy of parsing.

[0029] In the embodiment of the present application, through natural language processing technology, the system can extract specific complaints about power services from social media, such as "My home has a power outage today", and identify the time and location of the power outage. Combined with contextual information (such as weather conditions), the system can infer the possible cause of the power outage.

[0030] Furthermore, the present application proposes an improved solution to enhance the ability to understand unstructured data, aiming to more accurately extract topics and identify sentiment tendencies, thereby improving the performance of the entire intelligent decision-making system.

[0031] The options are as follows: Optionally, "the application of semantic understanding and context-aware technology to parse the unstructured data to extract themes and sentiment tendencies in the unstructured data" in step 102 includes: performing real-time processing on the received unstructured text data to obtain processed text data; performing vectorization processing on the vocabulary in the processed text data to obtain semantic relationship representation between vocabulary; performing entity information recognition processing on the processed text data to obtain entity information; performing sentence structure parsing processing on the processed text data to obtain grammatical dependencies between words; performing topic extraction processing on the processed text data based on the entity information and the grammatical dependencies between words to generate the main discussion themes or topics; performing context information processing on the processed text data to obtain context information at the sentence and paragraph levels, and performing sentiment tendency evaluation processing on the processed text data based on the context information at the sentence and paragraph levels to obtain sentiment tendency recognition results.

[0032] In an embodiment of the present application, unstructured text data is cleaned and preprocessed to remove noise and irrelevant information. The words in the text are converted into numerical vectors in order to calculate the semantic similarity between the words. Named entity recognition technology is used to extract key entities in the text, such as place names, time, events, etc. The grammatical dependencies between words in the text are analyzed to understand the structure of the sentences. Based on the entity information and grammatical dependencies, the main discussion topics or topics of the text are extracted. Contextual information at the sentence and paragraph levels is extracted for sentiment tendency assessment. Based on the contextual information, the sentiment tendency of the text is assessed, such as positive, negative or neutral.

[0033] In practical applications, suppose that in the field of energy and electricity, the system receives a user comment on social media: "There was another power outage at home today. It's so hot, it's really inconvenient." First, the system processes this comment in real time, removes irrelevant characters and punctuation marks, and obtains clean text data. Then, the words in the text are vectorized, and words such as "power outage", "weather", and "heat" are converted into vectors, and the semantic relationship between them is calculated. Next, named entity recognition technology is used to extract key entities such as "today" (time) and "home" (place). Subsequently, the sentence structure is analyzed to identify the grammatical dependency between "power outage" and "weather". Based on these entity information and grammatical dependencies, the system extracts that the main theme of the comment is "power outage problem". Finally, the system extracts contextual information at the sentence and paragraph levels and evaluates that the sentiment tendency of the comment is negative because the user expressed dissatisfaction and inconvenience with the power outage. This information can help power companies understand user feedback in a timely manner and take corresponding measures to solve the problem.

[0034] 103. Dynamically adjust the focus of data analysis according to the themes and sentiment tendencies, and use graph technology to build a correlation network between data; In this step, dynamic adjustment refers to adjusting the priority and direction of analysis according to changes in real-time data. Graph technology is used to build a relationship network between data entities to help discover implicit associations and patterns.

[0035] In the embodiment of the present application, when the system detects frequent power outage reports in a certain area, it will automatically increase the monitoring intensity of the area. At the same time, through the atlas technology, the system can analyze the relationship between the power grid structure of the area and other areas to find out potential weak links.

[0036] Furthermore, the present application proposes an improved solution to solve the above-mentioned problems and improve the flexibility and accuracy of data analysis by introducing a dynamic weight allocation mechanism and utilizing an external knowledge base to expand the association network.

[0037] The options are as follows: Optionally, the step 103 of "dynamically adjusting the focus of data analysis according to the topic and sentiment tendency, and using graph technology to construct an association network between data" includes: using a dynamic weight allocation mechanism to evaluate the importance of different data types contained in the topic and sentiment tendency to obtain a dynamic weight allocation strategy; according to the dynamic weight allocation strategy, adjusting the processing priority of different types of data in real time to obtain an adjusted data processing priority; based on the adjusted data processing priority, enhancing or weakening the processing depth of the corresponding real-time data stream to dynamically adjust the focus of data analysis; performing entity recognition processing on the unstructured data to obtain entities and their relationships; according to the entities and their relationships, using a graph construction algorithm, performing graph conversion processing on the entities and their relationships to generate a preliminary association network; performing pattern and trend discovery processing on the preliminary association network to identify indirect associations between nodes, and expanding the preliminary association network to generate an association network by introducing rules provided by an external knowledge base or domain experts.

[0038] In this scheme, the dynamic weight allocation mechanism refers to assigning different weights to different types of real-time data based on the importance and relevance of the data to reflect their importance in the analysis. The importance score is determined by evaluating the subject and sentiment tendency of the data. For example, negative sentiment data related to power failures will be given a higher weight. Data processing priority refers to adjusting the processing order of different types of data according to the dynamic weight allocation strategy to ensure that high-priority data is processed in a timely manner. Processing depth refers to the level of detail of data analysis. High-priority data will be analyzed more deeply. Entity recognition processing is to extract specific entities and their relationships from unstructured data through natural language processing technology. The graph construction algorithm is used to convert entities and their relationships into graph form to generate a preliminary association network. Pattern and trend discovery processing is to identify direct and indirect associations between data by analyzing nodes and edges in the graph, and further expand the association network. Rules provided by external knowledge bases or domain experts can be used to supplement and improve the association network to make it more comprehensive and accurate.

[0039] In the embodiment of the present application, first, the system uses a dynamic weight allocation mechanism to evaluate the importance of the data according to its theme and sentiment tendency, and generates a dynamic weight allocation strategy. Then, according to the dynamic weight allocation strategy, the processing priority of different types of data is adjusted in real time to ensure that high-priority data is processed in a timely manner. Next, based on the adjusted data processing priority, the system enhances or weakens the processing depth of the real-time data stream to dynamically adjust the focus of data analysis. At the same time, entity recognition processing is performed on unstructured data to extract entities and their relationships. Next, a graph construction algorithm is used to convert entities and their relationships into a graph form to generate a preliminary association network. Finally, the preliminary association network is processed for pattern and trend discovery, indirect associations between nodes are identified, and the final association network is generated by introducing rules provided by external knowledge bases or domain experts.

[0040] In the field of energy and power, assume that the system receives real-time data streams from multiple sources, including voltage and current data from the power grid monitoring system (structured data) and user comments on power services on social media (unstructured data). The system first uses a dynamic weight allocation mechanism to evaluate the importance of this data. For example, if negative comments about power outages are frequently detected in a certain area, these data will be given a higher weight. According to the dynamic weight allocation strategy, the system will prioritize these high-priority data and conduct more in-depth analysis on them. At the same time, the system will perform entity recognition processing on the comments on social media to extract the specific power outage location, time and user feedback. Next, the graph construction algorithm is used to convert these entities and their relationships into a graph form to generate a preliminary association network. Through the pattern and trend discovery processing of the preliminary association network, the system identifies the potential weak links of the power grid in the region and generates the final association network by introducing professional knowledge in the power field. Finally, based on these analysis results, the system recommends that the power company increase the frequency of power inspections in the region and deploy emergency power generation equipment in advance to prevent large-scale power outages.

[0041] 104. Analyze patterns and trends in the association network through a machine learning model, evaluate the mutual influence between different data entities, and predict future state changes to obtain prediction results; In this step, the machine learning model can learn patterns from large amounts of historical data and predict future development trends.

[0042] In an embodiment of the present application, by using historical electricity consumption data and weather information, a machine learning model can predict peak electricity consumption during the high temperatures of summer, as well as possible power shortages, and provide early warnings to power companies.

[0043] Furthermore, this application proposes an improvement scheme, which enhances the model's understanding ability by nodeizing key information of topics, sentiment tendencies, and structured data, and combining the relationships between these nodes, so as to more accurately evaluate the mutual influence between different data entities and improve the accuracy of future state predictions.

[0044] The options are as follows: Optionally, the step 104 of "analyzing the patterns and trends in the association network through a machine learning model, evaluating the mutual influence between different data entities, and predicting future state changes to obtain prediction results" includes: using the association network to link the relationships between the topics and sentiment tendencies and the noded relationships between key information in the structured data to obtain noded relationship links; analyzing and processing the patterns and trends in the association network based on the noded relationship links to obtain the trends of different data entities in the association network changing over time; based on the noded relationship links, evaluating and processing the mutual influence between the data entities in the association network, and training the machine learning model based on the trends of different data entities changing over time and the evaluation results; using the trained machine learning model to predict the future state based on the state of the association network in the current and past period of time and the noded relationship links, generating prediction values ​​for future state changes, and obtaining prediction results.

[0045] In this scheme, nodeization refers to converting topics, sentiment tendencies, and key information in structured data in the association network into nodes and establishing relationship links between these nodes. These nodes can be specific entities (such as a substation, user group) or abstract concepts (such as weather conditions, user satisfaction). The nodeization process helps to simplify complex data structures into a form that is easy to analyze.

[0046] The node-based relationship links refer to the connection relationships between nodes formed during the node-based process. These links can represent the associations between different data entities, such as the power supply relationship between substations and users, the causal relationship between weather conditions and power consumption, etc.

[0047] Trend analysis refers to analyzing the trends of different data entities over time based on node-based relationship links. For example, analyzing the changes in electricity consumption in a certain region in different seasons, the impact of weather conditions on electricity consumption, etc.

[0048] Mutual impact assessment refers to the evaluation of the interactions and impacts between different data entities based on node-based relationship links. For example, the evaluation of the impact of hot weather on the frequency of air conditioning use, the impact of user feedback on the service quality of power companies, etc.

[0049] A machine learning model is a mathematical model that uses algorithms to learn regularities and patterns from historical data. In this scenario, the machine learning model is used to learn patterns and trends in the association network, as well as the mutual influence between different data entities, to predict future state changes.

[0050] The predicted value refers to the estimated value of future state changes based on the current and past state of the associated network and the node-based relationship links through the trained machine learning model. For example, it predicts the peak power consumption in the next few days and the probability of power failure.

[0051] In an embodiment of the present application, first, the topics, sentiment tendencies, and key information of structured data in the association network are converted into nodes, and relationship links between these nodes are established to form a complex network structure; then, based on the node-based relationship links, the trends of different data entities changing over time are analyzed, such as changes in electricity consumption, the impact of weather conditions, etc.; then, the mutual influence between different data entities is evaluated, such as the impact of weather changes on electricity consumption, the impact of user feedback on grid stability, etc.; combined with the results of trend analysis and mutual influence evaluation, a machine learning model is trained to enable it to learn these complex relationships and patterns; finally, the trained model is used to predict future state changes based on the state of the association network and the node-based relationship links in the current and past period of time, and specific prediction values ​​are generated.

[0052] In the field of energy and electricity, suppose you need to predict the peak electricity consumption in a certain area during the high temperatures in summer. First, the system nodes the comments about electricity services on social media (such as "the air conditioner at home can't be turned on") and structured data (such as power grid monitoring data and meteorological data) to form an association network. Next, the system analyzes the changing trends of these nodes over time, such as the increase in electricity consumption in hot weather. Then, the mutual influence between different data entities is evaluated, such as the increase in air conditioning use due to hot weather, which increases electricity consumption. Based on these analysis results, a machine learning model is trained to enable it to predict the peak electricity consumption in hot weather. Finally, using the trained model, based on the current and past weather data and electricity consumption data, the peak electricity consumption in the next few days is predicted, and specific prediction values ​​are generated to help power companies prepare in advance and ensure the stability and reliability of power supply.

[0053] 105. Based on the pattern, the trend and the prediction result, generate a strategy recommendation for a specific application scenario; In this step, based on the analysis results, the system can make specific action recommendations to solve current or potential problems.

[0054] In an embodiment of the present application, for example, if it is predicted that the peak of summer electricity consumption is approaching in a certain area, the system recommends that the power company deploy mobile power generation vehicles in advance, or relieve the pressure by adjusting the load distribution of the power grid.

[0055] Furthermore, the present application proposes an improved solution to solve the above problems and improve the pertinence and practicality of strategic recommendations through more detailed trend analysis, identification of future event impacts, and multi-criteria decision analysis methods.

[0056] The options are as follows: Optionally, the "generating a policy recommendation for a specific application scenario based on the pattern, the trend and the prediction result" in step 105 includes: extracting key information points of the changing trend of different data entities in the associated network over time to obtain potential change rules; analyzing the prediction results, identifying changes in future events on the prediction results to obtain the impact of the changes on the current decision-making environment; defining an objective function based on the characteristics of the application scenario, quantifying the effectiveness and feasibility of the policy recommendation to obtain quantitative indicators; applying a multi-criteria decision analysis method in combination with the potential change rules, the impact of the changes on the current decision-making environment and the quantitative indicators to generate policy alternatives to obtain a set of policy alternatives; evaluating the performance of the set of policy alternatives in different scenarios to screen out the optimal policy recommendation.

[0057] In this scheme, key information point extraction processing refers to extracting important time nodes and features from the time-varying trends of different data entities in the associated network to reveal the underlying laws of change.

[0058] The change identification and processing of future events is to identify important events that may occur in the future and their changing trends by analyzing the prediction results, and to evaluate the impact of these changes on the current decision-making environment.

[0059] The objective function is a mathematical expression used to quantify the effectiveness and feasibility of a policy recommendation, usually including indicators such as cost, benefit, and risk.

[0060] Multi-criteria decision analysis is a decision-making tool that integrates multiple evaluation criteria to select the best option from multiple alternatives. Scenario evaluation refers to evaluating the performance of strategic alternatives under different assumptions to determine the best option.

[0061] In the embodiment of the present application, first, the system extracts key information points of different data entities changing over time from the associated network to reveal the underlying laws of change. Next, the system analyzes the prediction results, identifies the changing trends of future events, and evaluates the impact of these changes on the current decision-making environment. Then, based on the characteristics of the application scenario, the objective function is defined to quantify the effectiveness and feasibility of the strategy recommendations. Subsequently, the system applies a multi-criteria decision analysis method to generate multiple strategy alternatives. Finally, the system evaluates the performance of these alternatives in different scenarios and selects the optimal strategy recommendations.

[0062] In the field of energy and power, suppose the system detects a surge in electricity consumption in a certain area in summer and predicts that there may be a power shortage in the next few weeks. First, the system extracts key information points from historical electricity consumption data and finds that the peak of electricity consumption in summer is concentrated from July to September each year, and the higher the temperature, the greater the electricity consumption. Then, the system analyzes the prediction results and identifies that the temperature will continue to rise in the next few weeks, which may lead to a further increase in electricity consumption and put greater pressure on the current power grid load. Then, the system defines the objective function, considering indicators such as cost (the cost of starting the backup generator set), benefit (the economic benefit brought by ensuring stable power supply) and risk (power outage risk), and quantifies the effectiveness and feasibility of different strategies. Subsequently, the system applies multi-criteria decision analysis methods to generate multiple alternatives, such as starting the backup generator set in advance, adjusting the grid load distribution, and encouraging users to use electricity at off-peak times. Finally, the system evaluates the performance of these alternatives in different scenarios. For example, when the high temperature continues and the user response is different, the optimal strategy recommendation is finally selected, that is, starting the backup generator set in advance and combining the user's off-peak electricity consumption measures to ensure the stability and reliability of power supply.

[0063] 106. Convert the strategic recommendations into executable business rules to support intelligent decision making.

[0064] In this step, business rules refer to specific execution guidelines, which can be automatically triggered logic or manually executed instructions.

[0065] In an embodiment of the present application, for example, the power company can set, based on the system's recommendations, to automatically start a backup generator set when electricity consumption in a certain area reaches a preset threshold, or send power-saving reminders to users through smart meters to encourage off-peak electricity consumption.

[0066] Furthermore, the present application takes into account that in the existing big data intelligent decision-making analysis methods, although the importance scores of different data types can be evaluated through a dynamic weight allocation mechanism, there are some limitations in the actual application process. For example, the existing technology often relies on a fixed weight allocation strategy and cannot adapt to the changing data environment in real time; in addition, the single-dimensional weight allocation may ignore the various characteristics of the data source (such as time sensitivity, location relevance, etc.), resulting in important information being ignored or secondary information being over-processed. Therefore, the present application proposes an improved solution to calculate the importance score of the data type by introducing a more complex formula that takes into account multiple factors and their interactions to solve the above problems and improve the flexibility and accuracy of data analysis.

[0067] The options are as follows: Optionally, a dynamic weight allocation mechanism is used to evaluate the importance of different data types contained in the topic and sentiment tendency, including: The importance score of a data type is calculated using the following formula : ; in, It is The importance score of each data source, is the number of topics or sentiment dimensions considered, It is The time-dependent weight coefficient of each topic score, It is The data source is in The scores on the subject, It is The quadratic coefficient of the subject score is It is The time-dependent weight coefficient of the sentiment tendency score, It is The data source is in The scores on the emotional dimension, It is The absolute value coefficient of the sentiment score, is the time-dependent weight coefficient of the data source reliability score, It is The reliability score of each data source, is the distance factor, It is The Euclidean distance between each data source and the central data point, is the time sensitivity factor, It is Time correlation indicators for data sources, is the position sensitivity factor, It is Location relevance indicators for data sources, is the context sensitivity factor, It is Contextual consistency scores for data sources.

[0068] In big data intelligent decision-making analysis, different data sources have different importance and influence. Especially in the field of energy and power, the accuracy and timeliness of real-time data are crucial to decision-making. Therefore, a dynamic weight allocation mechanism is needed to evaluate the relative importance of different data types. This formula aims to comprehensively consider the topic score, sentiment tendency, reliability, spatiotemporal factors and contextual consistency of the data to calculate the importance score of each data source.

[0069] The overall design purpose of this formula is to comprehensively evaluate the importance and reliability of data sources and ensure that high-quality data can be fully utilized in the decision-making process. By introducing time-dependent weight coefficients, quadratic term coefficients, absolute value coefficients, etc., the formula can dynamically adjust the weights of different data types at different times and scenarios, thereby improving the accuracy and effectiveness of decision-making.

[0070] The following is a brief introduction to the design reasons of each sub-item of the formula: :This sub-item is mainly used to evaluate the scores of data sources on different topics and take into account time dependency and nonlinear effects. Time dependency weight coefficient This makes the topic scores at different time points have different weights, reflecting the dynamic changes in the importance of the topic. Used to capture the nonlinear effect of topic scores, i.e., some topics have a greater impact on the decision when they have high scores.

[0071] :This sub-item is mainly used to evaluate the scores of data sources on different sentiment dimensions, taking into account time dependency and sentiment intensity. Time dependency weight coefficient This makes the sentiment scores at different time points have different weights, reflecting the dynamic changes in the importance of sentiment tendency. It is used to emphasize the absolute value of sentiment tendency, that is, whether it is positive or negative sentiment, its strength will affect the importance score.

[0072] :This sub-item is mainly used to evaluate the reliability of the data source. Time dependence weight coefficient The reliability scores at different time points have different weights, reflecting the changes in the reliability of data sources over time. It directly reflects the quality and credibility of the data source.

[0073] : This sub-item is used to normalize the importance score, taking into account the spatial distance, temporal relevance, location relevance, and contextual consistency of the data source. Distance factor This reduces the importance of data sources that are farther away from the central data point. Time sensitivity factor This increases the importance of data sources with high time correlation. Location sensitivity factor This increases the importance of data sources with high location relevance. Context Sensitivity Factor This increases the importance of data sources with high context consistency.

[0074] The following is a brief introduction to how to obtain the various parameters of the formula: is the number of topics or sentiment dimensions considered: determined through expert evaluation and historical data analysis. For example, a power company can determine the main topics of concern (e.g., voltage stability, customer satisfaction, weather impact) and sentiment dimensions (e.g., positive sentiment, negative sentiment) by analyzing past fault reports, customer feedback, and weather data.

[0075] It is Time-dependent weight coefficients for each topic score: Determined through historical data analysis and expert evaluation. For example, a power company can determine the importance of each topic in different time periods by analyzing the changes in topic scores at different time points (such as summer and winter) over the past few years. Time series analysis methods (such as ARIMA models) can be used to predict future time-dependent weights.

[0076] It is The data source is in Scores on topics are obtained through natural language processing (NLP) technology and expert evaluation. For example, for data from power grid monitoring systems, NLP technology can be used to extract keywords and phrases related to voltage stability from monitoring reports, and then scored through expert evaluation. For social media data, sentiment analysis tools can be used to extract comments related to user satisfaction and score them.

[0077] It is The quadratic coefficient of the subject score is determined through experiments and statistical analysis. For example, a power company can determine the quadratic coefficient by analyzing the impact of different subject scores on decision-making through experiments. Regression analysis methods can be used to fit the relationship between subject scores and decision-making effects to determine the appropriate quadratic coefficient.

[0078] It is The time-dependent weight coefficient of the sentiment tendency score is similar to , determined through historical data analysis and expert evaluation. For example, a power company can determine the importance of each sentiment dimension in different time periods by analyzing the changes in sentiment tendency scores at different time points (such as summer and winter) in the past few years. Time series analysis methods (such as ARIMA models) can be used to predict future time-dependent weights.

[0079] It is The data source is in Scores on the sentiment dimension: Determined by sentiment analysis tools and expert evaluation. For example, for social media data, sentiment analysis tools (such as VADER and TextBlob) can be used to extract sentiment in comments and score them. For user feedback, NLP technology can be used to extract keywords and phrases, and then expert evaluation can be used to score them.

[0080] It is The absolute value coefficient of each sentiment score: determined through experiments and statistical analysis. For example, a power company can analyze the impact of the absolute values ​​of different sentiment scores on decision-making through experiments to determine the absolute value coefficient. Regression analysis methods can be used to fit the relationship between sentiment scores and decision-making effects to determine the appropriate absolute value coefficient.

[0081] It is the time-dependent weight coefficient of the data source reliability score: determined by historical data analysis and expert evaluation. For example, a power company can determine the reliability of each data source in different time periods by analyzing the changes in data source reliability at different time points (such as summer and winter) in the past few years. Time series analysis methods (such as ARIMA models) can be used to predict future time-dependent weights.

[0082] It is Reliability score of each data source: determined by historical data quality and user feedback evaluation. For example, for the data of the power grid monitoring system, the accuracy and completeness of its historical data and the feedback from users on its data quality can be analyzed. For the weather forecast data of the meteorological station, the accuracy and timeliness of its forecast can be analyzed.

[0083] Distance factor: determined through experiments and statistical analysis. For example, a power company can determine the distance factor by analyzing the impact of data sources at different distances on decision-making through experiments. Regression analysis methods can be used to fit the relationship between distance and decision-making effect to determine the appropriate distance factor.

[0084] It is Euclidean distance between each data source and a central data point: calculated using Geographic Information System (GIS) data. For example, a power company can use GIS tools to calculate the Euclidean distance between each data source and a central data point, such as the power company headquarters.

[0085] It is a time sensitivity factor: determined through experiments and statistical analysis. For example, a power company can determine the time sensitivity factor by analyzing the impact of data sources with different time sensitivities on decision-making through experiments. Regression analysis methods can be used to fit the relationship between time sensitivity and decision-making effect, thereby determining the appropriate time sensitivity factor.

[0086] It is Time correlation index of each data source: determined by time series analysis. For example, a power company can use time series analysis methods (such as autocorrelation function ACF, partial autocorrelation function PACF) to analyze the time correlation of each data source and determine the time correlation index.

[0087] It is a location sensitivity factor: determined through experiments and statistical analysis. For example, a power company can determine the location sensitivity factor by analyzing the impact of data sources in different locations on decision-making through experiments. Regression analysis methods can be used to fit the relationship between location sensitivity and decision-making effect, thereby determining the appropriate location sensitivity factor.

[0088] It is Location relevance indicators for each data source: Determined through geographic information system (GIS) data and spatial analysis methods. For example, a power company can use GIS tools to analyze the relevance of each data source in a specific geographic location to determine the location relevance indicator.

[0089] It is a context sensitivity factor: determined through experiments and statistical analysis. For example, a power company can determine the context sensitivity factor by analyzing the impact of data sources with different contexts on decision-making through experiments. Regression analysis methods can be used to fit the relationship between context sensitivity and decision-making effect, thereby determining the appropriate context sensitivity factor.

[0090] It is Contextual consistency scores for each data source: determined through expert evaluation and contextual analysis tools. For example, a power company can use contextual analysis tools (such as topic modeling and cluster analysis) to analyze the relevance and consistency of each data source in a specific context, and then score it through expert evaluation.

[0091] In the embodiment of the present application, it is assumed that a power company needs to evaluate the importance of multiple data sources during the peak period of summer electricity consumption in order to optimize power dispatch. The data sources include: Data source 1: Real-time voltage and current data from the power grid monitoring system.

[0092] Data source 2: User feedback on social media.

[0093] Data source 3: weather forecast data from weather stations.

[0094] The parameter settings are as follows: :The topics and sentiment dimensions considered are voltage stability, user satisfaction, and weather impact.

[0095] : The time-dependent weight coefficient of the topic score.

[0096] : The time-dependent weight coefficient of sentiment tendency score.

[0097] : The quadratic coefficient of the topic score.

[0098] : Absolute value coefficient of sentiment score.

[0099] : The time-dependent weight coefficient of the data source reliability score.

[0100] : Distance factor, time sensitivity factor, location sensitivity factor, context sensitivity factor.

[0101] : The coffeeability score of the data source.

[0102] : The Euclidean distance between the data source and the central data point.

[0103] : Time correlation indicator of data source.

[0104] : Location relevance indicator of the data source.

[0105] : The contextual consistency score of the data source.

[0106] Data Source Score: :Scores of data source 1 on voltage stability, user satisfaction, and weather impact.

[0107] :Scores of data source 2 on voltage stability, user satisfaction, and weather impact.

[0108] :Scores of data source 3 on voltage stability, user satisfaction, and weather impact.

[0109] : Sentiment scores of data source 1 on voltage stability, user satisfaction, and weather impact.

[0110] : Sentiment scores of data source 2 on voltage stability, user satisfaction, and weather impact.

[0111] : Sentiment scores of data source 3 on voltage stability, user satisfaction, and weather impact.

[0112] The calculation process is as follows: Importance score of data source 1 : ; After substituting the values: ; Importance score of data source 2 : ; After substituting the values: ; Importance score of data source 3 : ; After substituting the values: .

[0113] The importance score of data source 1 (real-time voltage and current data from the power grid monitoring system) is 4.35, indicating that its data on voltage stability, user satisfaction and weather impact are very critical and highly reliable.

[0114] The importance score of data source 2 (user feedback on social media) is 2.53, indicating that although user feedback has certain value, it is not as reliable, time-sensitive, and location-relevant as other data sources.

[0115] The importance score of data source 3 (weather forecast data from meteorological stations) is 3.41, indicating that weather forecast data has high value in predicting and responding to summer peak electricity consumption, especially in terms of weather impact.

[0116] In summary, power companies should focus on the real-time data of the power grid monitoring system during the peak period of electricity consumption in summer, and refer to the weather forecast data of the meteorological station to optimize power dispatch and ensure stable power supply. User feedback on social media can be used as auxiliary information to understand user needs and improve service quality.

[0117] Furthermore, the present application takes into account that in the existing big data intelligent decision-making analysis methods, although the trained machine learning model can be used to predict the future state, there are some limitations in the actual application process. For example, the existing technology often relies only on a single performance indicator (such as absolute error or mean square error) to evaluate the prediction effect of the model, which may lead to an incomplete understanding of the overall performance of the model. In addition, for complex systems, a single indicator may not be able to fully reflect the performance of the model in different aspects, such as causal relationship identification, external factor influence, and similarity of graph structure. Therefore, the present application proposes an improved scheme to calculate the overall prediction performance of the model by introducing a comprehensive formula. The formula takes into account performance indicators of multiple dimensions and their interactions to solve the above problems and improve the reliability and practicality of the prediction results.

[0118] The options are as follows: Optionally, the method uses the trained machine learning model to perform prediction processing of future states according to the states of the associated networks in the current and past period of time and the node-based relationship links, generates prediction values ​​about future state changes, and obtains prediction results, including: The following formula is used to combine the machine learning model to generate predictions about future state changes and obtain the prediction results : ; in, is an evaluation indicator of the overall prediction performance of the model. is the number of test samples, is the time point or sample number, It's time point The absolute error between the predicted result of the above model and the true value, is the accuracy of causal inference, at the time point The degree to which the above model can correctly identify causal relationships, is variability, It's time point On The actual value of the external factor, is the predicted value of the kth external factor at time point t, is the number of external factors considered, It is an external factor The influence weight of is the graph similarity index, They are the absolute error over time, the accuracy of causal inference, the influence of variability and external factors, and the weight coefficient of graph similarity, which also need to satisfy conditions, is the decay rate of absolute error, It is variability The standard deviation of .

[0119] In complex systems, especially those involving multiple variables and external factors, such as the energy and power sector, accurate prediction of future states is critical to optimizing operations and management. Machine learning models can use historical data and current states to predict future state changes. However, a single prediction indicator may not fully reflect the performance of the model, so multiple indicators need to be combined to evaluate the overall prediction performance of the model. This formula generates a prediction value about future state changes by combining multiple indicators to evaluate the overall prediction performance of the model.

[0120] The overall design purpose of this formula is to comprehensively evaluate the performance of machine learning models in predicting future state changes. By integrating multiple key indicators, including absolute error, causal inference accuracy, variability, external factors, and graph similarity, the predictive ability and robustness of the model can be more comprehensively reflected. Each indicator has its own unique meaning, and the impact of different indicators on the overall performance can be balanced by weighted averaging.

[0121] The following is an introduction to the design reasons of each item in the formula: : This sub-item is used to evaluate the absolute error between the model prediction result and the true value. Exponential decay function This makes the prediction results with smaller errors contribute more to the overall performance, while the negative impact of larger errors on the overall performance will gradually weaken. is the absolute difference between the model prediction and the true value at time point t, is the decay rate of the absolute error.

[0122] : This sub-item is used to evaluate the causal inference accuracy of the model. Logarithmic function This makes predictions with higher causal inference accuracy contribute more to the overall performance. It indicates the degree to which the model can correctly identify causal relationships at time point t.

[0123] : This sub-item is used to evaluate the variability of the model's prediction results. Standard deviation It reflects the volatility of the forecast results. Higher variability may mean greater uncertainty in the model. , can balance the impact of variability on the overall performance.

[0124] :This sub-item is used to evaluate the impact of external factors on the model prediction results. Product form This allows the impact of each external factor’s prediction error on overall performance to be taken into account. is the actual value of the kth external factor at time point t, is the predicted value of the kth external factor at time point t, is the influence weight of external factor k.

[0125] :This sub-item is used to evaluate the similarity between the model prediction results and the actual graph. Graph similarity index It reflects the matching degree between the predicted result and the actual map. The higher the similarity, the closer the predicted result of the model is to the actual situation. , which can balance the impact of graph similarity on the overall performance.

[0126] The following is an introduction to how to obtain the parameters of the formula: N is the number of test samples: It is determined by experimental design, usually by cross-validation or independent test sets. For example, the historical electricity consumption data can be divided into a training set and a test set, and the number of samples in the test set is N.

[0127] is a time point or sample number: determined by the timestamp or sample number of the data. For example, if the data is recorded hourly, t is the time point of each hour.

[0128] It's time point The absolute error between the model prediction result and the true value is obtained by calculating the absolute difference between the model prediction value and the true value. For example, if the model predicts that the electricity consumption in a certain hour is 1000 kWh and the actual electricity consumption is 1050 kWh, then At=|1000−1050|=50At=|1000−1050|=50 kWh.

[0129] is the accuracy of causal inference, at the time point The degree to which the above model can correctly identify causal relationships: The accuracy of the model's causal inference at time point t is evaluated through causal inference algorithms (such as causal Bayesian networks and causal forests). For example, a causal Bayesian network can be used to analyze the impact of external factors such as temperature and humidity on electricity consumption, and to evaluate whether the model correctly identifies these causal relationships.

[0130] Variability: obtained by calculating the standard deviation of the model prediction results. For example, the standard deviation of the prediction results of multiple prediction models at the same time point t can be calculated to evaluate the variability of the model.

[0131] It's time point On Actual values ​​of external factors: extracted from historical data. For example, historical data of temperature, humidity, and wind speed can be obtained from a weather station.

[0132] It's time point On Predicted values ​​of external factors: obtained through external factor prediction models (such as time series prediction models). For example, the ARIMA model can be used to predict temperature, humidity, and wind speed in the next few hours.

[0133] is the number of external factors considered: determined by the complexity of the problem and the available data. For example, common external factors include temperature, humidity, wind speed, holidays, working days, etc.

[0134] It is an external factor Impact weight: Determined through expert evaluation or statistical analysis. For example, the impact of each external factor on electricity consumption can be evaluated through regression analysis to determine its weight.

[0135] It is a graph similarity indicator: the similarity between the model prediction result and the actual graph is evaluated through the graph matching algorithm (such as graph edit distance and graph kernel method). For example, the graph edit distance algorithm can be used to compare the similarity between the predicted power grid topology and the actual power grid topology.

[0136] They are the absolute error over time, causal inference accuracy, variability and external factors, and weight coefficients of graph similarity: determined through historical data analysis and expert evaluation to ensure that they meet For example, the importance of each indicator at different time points can be determined through performance evaluation of historical data, so as to adjust the weight coefficient.

[0137] is the decay rate of the absolute error: determined experimentally and statistically, the optimal value is usually found through grid search or random search. For example, you can use cross-validation to try different value, choose the one that gives the best model performance value.

[0138] In the embodiment of the present application, a power company needs to predict the power demand in the next few weeks during the peak period of summer power consumption in order to optimize power dispatch and management. To this end, the company uses a trained machine learning model to predict the future state based on the state of the associated network and the node-based relationship links in the current and past period of time, generates a predicted value for the future state change, and obtains a prediction result.

[0139] First, prepare the data: Collect historical electricity usage data, including hourly electricity usage, temperature, humidity, wind speed and other external factors.

[0140] Collect power grid monitoring data, including real-time monitoring data such as voltage and current.

[0141] Collect user feedback data, including user satisfaction, complaints and other information.

[0142] The data is divided into a training set and a test set. The number of samples in the test set is N.

[0143] Second, perform model training: Use the training set data to train the machine learning model, such as using a long short-term memory network (LSTM) or a support vector machine (SVM) model. During the training process, adjust the model's hyperparameters to improve the model's predictive performance.

[0144] Furthermore, the prediction results are generated: Using the trained model, the predicted value of the future state is generated according to the state of the associated network and the node-based relationship links in the current and past period of time.

[0145] For each time point t, calculate the absolute error between the model prediction result and the true value .

[0146] Calculate the causal inference accuracy of the model at time point t .

[0147] Calculate the variability of the model at time point t .

[0148] Calculate the actual value of each external factor and predicted values .

[0149] Calculating graph similarity metrics .

[0150] Finally, a performance evaluation is performed: Calculate the comprehensive evaluation value at each time point t: ; Calculate the evaluation index P of the overall prediction performance of the model: ; In the embodiment of the present application, the accuracy and robustness of the model are evaluated by analyzing the prediction performance of the model at different time points. According to the evaluation results, the model parameters and the influence weights of external factors are optimized to further improve the prediction performance of the model. The prediction results are applied to power dispatching and management to optimize power supply and ensure the stability and reliability of power supply during the peak period of summer power consumption.

[0151] Through the above embodiments, power companies can use machine learning models and comprehensive evaluation formulas to effectively predict future electricity demand and optimize power dispatch and management. This method not only improves the accuracy of predictions, but also enhances the robustness and reliability of the model, helping power companies better cope with the challenges of peak summer electricity demand.

[0152] Figure 2 A structural diagram of a big data intelligent decision analysis system based on machine learning is provided for the embodiment of the present application, such as Figure 2 As shown, the device comprises: A receiving module 21, configured to receive real-time data streams from multiple sources, wherein the real-time data streams include structured data and unstructured data; An extraction module 22, used to parse the unstructured data using semantic understanding and context awareness techniques to extract topics and sentiment tendencies from the unstructured data; An adjustment module 23, used to dynamically adjust the focus of data analysis according to the theme and sentiment tendency, and to construct an association network between data using graph technology; An evaluation module 24 is used to analyze patterns and trends in the association network through a machine learning model, evaluate the mutual influence between different data entities, and predict future state changes to obtain prediction results; A generating module 25, configured to generate a strategy suggestion for a specific application scenario based on the pattern, the trend and the prediction result; The conversion module 26 is used to convert the policy recommendations into executable business rules to support intelligent decision making.

[0153] Figure 2 The big data intelligent decision analysis system based on machine learning can perform Figure 1The implementation principle and technical effect of the big data intelligent decision analysis method based on machine learning described in the illustrated embodiment will not be repeated. The specific manner in which each module and unit performs operations in the big data intelligent decision analysis system based on machine learning in the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated here.

[0154] In one possible design, Figure 2 A big data intelligent decision analysis system based on machine learning in the illustrated embodiment can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32; The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0155] The processing component 32 is used to: receive real-time data streams from multiple sources, the real-time data streams including structured data and unstructured data; apply semantic understanding and context-aware technology to parse unstructured data to extract themes and sentiment tendencies in unstructured data; dynamically adjust the focus of data analysis, and use graph technology to build an association network between data; evaluate the mutual influence between different data entities, and predict future state changes to obtain prediction results; generate policy recommendations for specific application scenarios based on patterns, trends and prediction results; convert policy recommendations into executable business rules to support intelligent decision making.

[0156] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0157] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0158] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0159] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0160] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0161] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0162] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a big data intelligent decision analysis method based on machine learning.

[0163] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0164] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0165] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A big data intelligent decision analysis method based on machine learning, characterized in that: include: Receiving real-time data streams from multiple sources, the real-time data streams including structured data and unstructured data; Applying semantic understanding and context awareness technology to parse the unstructured data to extract topics and sentiment tendencies in the unstructured data; Dynamically adjust the focus of data analysis based on the topic and sentiment, and use graph technology to build a network of associations between data; Analyze patterns and trends in the association network through machine learning models, evaluate the mutual influence between different data entities, and predict future state changes to obtain prediction results; Based on the pattern, the trend and the prediction result, generate strategy recommendations for specific application scenarios; Translate the policy recommendations into executable business rules to support intelligent decision making.

2. The method according to claim 1, characterized in that The focus of data analysis is dynamically adjusted according to the topic and sentiment tendency, and a correlation network between data is constructed using graph technology, including: Using a dynamic weight allocation mechanism, different data types included in the topic and sentiment tendency are evaluated for importance scoring, and a dynamic weight allocation strategy is obtained; According to the dynamic weight allocation strategy, the processing priorities of different types of data are adjusted in real time to obtain adjusted data processing priorities; Based on the adjusted data processing priority, the processing depth of the corresponding real-time data stream is enhanced or weakened to dynamically adjust the focus of data analysis; Performing entity recognition processing on the unstructured data to obtain entities and their relationships; According to the entities and their relationships, a graph construction algorithm is used to perform graph conversion processing on the entities and their relationships to generate a preliminary association network; The preliminary association network is processed for pattern and trend discovery to identify indirect associations between nodes, and the preliminary association network is processed for expansion to generate an association network by introducing rules provided by an external knowledge base or domain experts.

3. The method according to claim 2, characterized in that The machine learning model is used to analyze patterns and trends in the association network, evaluate the mutual influence between different data entities, and predict future state changes to obtain prediction results, including: Using the association network, link processing is performed on the relationship between the topics and sentiment tendencies and the node-based key information in the structured data to obtain node-based relationship links; Analyzing and processing the patterns and trends in the association network according to the node-based relationship links to obtain the trends of changes of different data entities in the association network over time; Based on the node-based relationship links, the mutual influence between the data entities in the association network is evaluated and processed, and the machine learning model is trained based on the trends of different data entities changing over time and the evaluation results; By using the trained machine learning model, the future state is predicted based on the state of the associated network and the node-based relationship links in the current and past period of time, and the predicted value of the future state change is generated to obtain the prediction result.

4. The method according to claim 2, characterized in that: Using a dynamic weighting mechanism, different data types contained in the topic and sentiment are evaluated for importance scoring, including: The importance score of a data type is calculated using the following formula : ； in, It is The importance score of each data source, is the number of topics or sentiment dimensions considered, It is The time-dependent weight coefficient of each topic score, It is The data source is in The scores on the subject, It is The quadratic coefficient of the subject score is It is The time-dependent weight coefficient of the sentiment tendency score, It is The data source is in The scores on the emotional dimension, It is The absolute value coefficient of the sentiment score, is the time-dependent weight coefficient of the data source reliability score, It is The reliability score of each data source, is the distance factor, It is The Euclidean distance between each data source and the central data point, is the time sensitivity factor, It is Time correlation indicators for data sources, is the position sensitivity factor, It is Location relevance indicators for data sources, is the context sensitivity factor, It is Contextual consistency scores for data sources.

5. The method according to claim 3, characterized in that: The trained machine learning model is used to perform prediction processing of the future state according to the state of the associated network in the current and past period of time and the node-based relationship links, generate a prediction value about the future state change, and obtain a prediction result, including: The following formula is used to combine the machine learning model to generate predictions about future state changes and obtain the prediction results : ； in, is an evaluation indicator of the overall prediction performance of the model. is the number of test samples, is the time point or sample number, It's time point The absolute error between the predicted result of the above model and the true value, is the accuracy of causal inference, at the time point The degree to which the above model can correctly identify causal relationships, is variability, It's time point On The actual value of the external factor, It's time point On The predicted value of an external factor, is the number of external factors considered, It is an external factor The influence weight of is the graph similarity index, They are the absolute error over time, the accuracy of causal inference, the influence of variability and external factors, and the weight coefficient of graph similarity, which also need to satisfy conditions, is the decay rate of absolute error, It is variability The standard deviation of .

6. The method according to claim 1, characterized in that The application of semantic understanding and context awareness technology to parse the unstructured data to extract topics and sentiment tendencies in the unstructured data includes: Process the received unstructured text data in real time to obtain processed text data; Performing vectorization processing on the words in the processed text data to obtain a semantic relationship representation between the words; Performing entity information recognition processing on the processed text data to obtain entity information; Performing sentence structure analysis on the processed text data to obtain grammatical dependency relationships between words; Based on the entity information and the grammatical dependency relationship between words, subject extraction processing is performed on the processed text data to generate the main discussion subject or topic; The processed text data is subjected to context information processing to obtain context information at the sentence and paragraph levels, and the processed text data is subjected to sentiment tendency assessment processing based on the context information at the sentence and paragraph levels to obtain a recognition result of the sentiment tendency.

7. The method according to claim 3, characterized in that The generating of a strategy suggestion for a specific application scenario based on the pattern, the trend and the prediction result includes: Extract key information points from the time-varying trends of different data entities in the association network to obtain potential change rules; Analyze the prediction results, identify changes of future events on the prediction results, and obtain the impact of the changes on the current decision-making environment; According to the characteristics of the application scenario, define the objective function, quantify the effectiveness and feasibility of the strategy recommendations, and obtain quantitative indicators; Combining the potential change rules, the impact of the change on the current decision-making environment and the quantitative indicators, a multi-criteria decision analysis method is applied to generate and process the strategic alternatives to obtain a set of strategic alternatives; The performance of the strategy alternatives set in different scenarios is evaluated and processed to screen out the optimal strategy recommendation.

8. A big data intelligent decision analysis system based on machine learning, characterized in that: include: A receiving module, used for receiving real-time data streams from multiple sources, wherein the real-time data streams include structured data and unstructured data; An extraction module, used for applying semantic understanding and context awareness technology to parse the unstructured data to extract themes and sentiment tendencies in the unstructured data; An adjustment module, used to dynamically adjust the focus of data analysis according to the topic and sentiment tendency, and to construct an association network between data using graph technology; An evaluation module, used to analyze patterns and trends in the association network through a machine learning model, evaluate the mutual influence between different data entities, and predict future state changes to obtain prediction results; A generation module, used to generate a strategy suggestion for a specific application scenario based on the pattern, the trend and the prediction result; The conversion module is used to convert the policy recommendations into executable business rules to support intelligent decision making.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a big data intelligent decision analysis method based on machine learning as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a big data intelligent decision analysis method based on machine learning is implemented as described in any one of claims 1 to 7.

Images