Method, system and equipment for generating data analysis report and medium

Abstract

The invention is suitable for the technical field of computers, and provides a data analysis report generation method, which comprises the following steps: obtaining an analysis demand of a user, and performing intention analysis on the analysis demand to obtain a preliminary research demand; planning a preliminary layout of an analysis report based on the preliminary research demand; based on the preliminary layout, decomposing the preliminary research demand into a plurality of research tasks; and for each research task, in combination with the content of the knowledge base, respectively carrying out deep research, generating an analysis draft, carrying out global integration and correlation analysis on each analysis draft, and compiling a complete data analysis report. A unified data conversion and data analysis process is provided, and the data integration and comprehensive analysis capability is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computers, and particularly relates to a data analysis report generation method, system, device and medium. BACKGROUND

[0002] In today's era of rapid development of digital information, data analysis has become a key basis for enterprise decision-making, market forecasting, and business optimization in many fields. Efficient, accurate and in-depth data analysis reports are increasingly important to various organizations.

[0003] The existing data analysis report generation method is usually generated manually. When processing massive data, manual analysis faces huge time and energy consumption. With the exponential growth of data volume, manual methods are difficult to complete comprehensive sorting and analysis of all data within a reasonable time. In addition, for real-time analysis needs, manual analysis is not up to the task. In addition, in the traditional manual analysis process, the analysis results are highly dependent on the personal experience, knowledge background and interpretation perspective of the analyst. Different analysts may draw completely different conclusions from the same set of data due to differences in their own experiences and cognition. This subjectivity can easily lead to systematic bias, making the analysis results unable to objectively and accurately reflect the true situation of the data. Moreover, analysts may be influenced by their own thinking patterns and overlook key but "unobvious" data patterns, affecting the comprehensiveness and accuracy of the data analysis report. Furthermore, in actual data analysis work, different analysts or teams often use their own habitual report formats, analysis dimensions and terminology systems. This difference makes it difficult to compare data analysis reports from different sources, and it is difficult to form a unified standard and specification. This not only increases the difficulty of data integration and comprehensive analysis, but also reduces the readability and understandability of data analysis reports, which is not conducive to knowledge sharing and decision-making within the organization.

[0004] In summary, the existing manual data analysis report generation method has many problems in efficiency, accuracy, consistency, and timeliness, and cannot meet the needs of modern data-driven decision-making. Therefore, there is an urgent need for a new data analysis report generation method to solve the above problems and improve the quality and value of data analysis reports. SUMMARY

[0005] The embodiments of the present application provide a data analysis report generation method, system, device and medium, which can solve one of the above technical problems.

[0006] In a first aspect, the embodiments of the present application provide a data analysis report generation method, comprising: Obtaining the analysis requirements of the user, and performing intent analysis on the analysis requirements to obtain preliminary research requirements; Based on the preliminary research needs, a preliminary layout of the analysis report is planned; Based on the preliminary layout, the preliminary research needs are decomposed into a plurality of research tasks; For each of the research tasks, in-depth research is conducted in combination with the knowledge base content, and an analysis draft is generated; Each of the analysis drafts is globally integrated and correlation analyzed, and a complete data analysis report is written.

[0007] Further, the analysis needs of the user are obtained, and the analysis needs are intention parsed to obtain preliminary research needs, including: The analysis needs of the user are input into an intention query model, and analysis entities are extracted; The analysis entities are matched by a rule engine to determine analysis intentions; Based on the analysis entities, search and matching are performed in the knowledge base content to obtain analysis metadata; The analysis intentions and the analysis metadata are stored in a structured form to generate preliminary research needs.

[0008] Further, based on the preliminary research needs, a preliminary layout of the analysis report is planned, including: Based on the analysis entities, an entity graph is constructed, and the correlation strength between different entities is calculated; Based on the correlation strength, the analysis needs are classified to determine a plurality of analysis topics; Based on the analysis topics, in combination with the analysis intentions, an analysis report framework is determined to generate a preliminary layout.

[0009] Further, for each of the research tasks, in-depth research is conducted in combination with the knowledge base content to generate an analysis draft, including: Based on the research tasks, knowledge retrieval is performed from the knowledge base content to obtain a plurality of source data sets; Based on the plurality of source data sets, in combination with the analysis intentions, data analysis is performed; The results of the data analysis are filled into the analysis report framework to generate an analysis draft.

[0010] Further, for each of the research tasks, in-depth research is conducted in combination with the knowledge base content to generate an analysis draft, including: For a plurality of research tasks, a task scheduling algorithm is used to allocate thread resources to generate a thread allocation scheme; Data streams in the research process are obtained, and the data streams are allocated into a plurality of data blocks to generate an initial data block set; For each initial data block set, a digital watermark embedding algorithm is used to assign a unique identifier to each data block to generate a watermark data block set; During the research process, the unique identifier of each data block is extracted from the watermark data block set for integrity verification, and parallel processing of multiple research tasks is performed based on a thread allocation scheme.

[0011] Further, the global integration and correlation analysis of each analysis draft include: For each analysis draft, the analysis draft is converted into structured information to build an evidence support network; Based on the evidence support network of each analysis draft, multi-source data fusion is performed, and correlation analysis of the analysis theme is performed.

[0012] Further, the writing of a complete data analysis report includes: Based on the analysis intent, a core narrative framework is selected, a pre-set corpus is combined, an introduction is added to the data analysis report, a conclusion is written, suggestions are added, and references are sorted; A text optimization algorithm is used to adjust the order of sentences and conjunctions to generate a logically clear data analysis report.

[0013] In a second aspect, the embodiments of the present application provide a data analysis report generation system, which includes: A first processing module is configured to obtain an analysis requirement of a user and perform intent analysis on the analysis requirement to obtain a preliminary research requirement; A second processing module is configured to plan a preliminary layout of an analysis report based on the preliminary research requirement; A third processing module is configured to decompose the preliminary research requirement into multiple research tasks based on the preliminary layout; A fourth processing module is configured to conduct in-depth research on each research task based on the knowledge base content to generate an analysis draft; A fifth processing module is configured to globally integrate and perform correlation analysis on each analysis draft and write a complete data analysis report.

[0014] In a third aspect, the embodiments of the present application provide a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the data analysis report generation method described above when executing the computer program.

[0015] In a fourth aspect, the embodiments of the present application provide a computer readable storage medium, which includes a computer program stored in the computer readable storage medium, and the computer program is executed by a processor to implement the data analysis report generation method described above.

[0016] The beneficial effects of the embodiments of the present application compared with the prior art are: The present application discloses a data analysis report generation method, which analyzes the user's analysis needs, eliminates the ambiguity of natural language expression, ensures that the research needs are highly matched with the user's real intention, eliminates the analysis errors caused by the differences in experience and cognition of different analysts, further allocates the theme direction of the user's research needs, plans the preliminary layout of the analysis report, i.e., determines the research direction of the analysis report, thereby objectively studies and analyzes the data, and on this basis, maps it into multiple research tasks, improves the research efficiency of the data analysis report, and thus, the present application provides a unified data transformation and data analysis process from data collection, collation, analysis to report writing process, improves the data integration and comprehensive analysis capability, and takes the analysis intention as the core to select a suitable core narrative framework for the data analysis report, thereby improving the rationality, readability and understandability of the data analysis report. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a flow diagram of a data analysis report generation method provided by an embodiment of the present application; Figure 2 is a structural diagram of a data analysis report generation system provided by an embodiment of the present application; Figure 3 is a structural diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0019] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments without these specific details. In other instances, well-known systems, devices, circuits and methods have not been described in detail in order to avoid obscuring the description of the present application.

[0020] It should be understood that the word “comprise” or variations such as “comprises” or “comprising”, when used in this specification and in the accompanying claims, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0021] It should also be understood that the term “and / or” when used in this specification and in the following claims is to be interpreted as to include one or more of the listed items, as well as any combination of the items.

[0022] As used in this specification and in the claims, the terms “if’ and “when” can be interpreted to mean “upon” or “in response to a determination” or “in response to a detection” depending on the context. Similarly, the phrase “if it is determined” or “if [a described condition or event] is detected” can be interpreted to mean “upon a determination” or “in response to a determination” or “upon detecting [a described condition or event]” or “in response to detecting [a described condition or event]”, depending on the context.

[0023] In addition, in the description of the application and in the following claims, the terms “first”, “second”, “third”, etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0024] Reference in the specification to “one embodiment” or “some embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, the appearances of the phrases “in one embodiment”, “in some embodiments”, “in other embodiments”, “in additional embodiments”, and so on, in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specifically stated. The terms “comprise”, “comprises”, “comprising”, “include”, “includes”, “including” and their variants are meant to be synonymous with “including but not limited to”, unless otherwise specifically stated.

[0025] See Figure 1 As shown in the drawings, the application is a method for generating a data analysis report, comprising the following steps: S100, obtaining the analysis requirements of the user, and performing intent analysis on the analysis requirements to obtain preliminary research requirements; In this application, the intent analysis on the analysis requirements of the user eliminates the ambiguity of natural language expression, ensures that the research requirements are highly matched with the real intention of the user, and eliminates the analysis errors caused by the differences in experience and cognition of different analysts.

[0026] In some embodiments, the above step S100 comprises: inputting an analysis requirement of a user into an intent query model, extracting an analysis entity; performing rule engine matching on the analysis entity, determining an analysis intent; searching and matching in a knowledge base content based on the analysis entity, obtaining analysis metadata; storing the analysis intent and the analysis metadata in a structured form, generating a preliminary research requirement.

[0027] In this embodiment, the analysis requirement text input by the user is processed by the preset intent query model to obtain the corresponding analysis entity, specifically, the intent query model is a natural language processing model such as BERT or GPT series model, which is trained on a large-scale text corpus in an unsupervised manner, so as to capture rich language semantic information, specifically, the data related to data analysis query is collected, a data set containing user query and corresponding label graph is constructed, and the intent query model is further trained by using the data set, so as to better adapt to the intent analysis task in the field of data analysis.

[0028] In one possible embodiment, the user inputs the analysis requirement text in the form of natural language, for example, "Q3 sales in East China region decline reason", in this process, the intent query model encodes the input analysis requirement text in the form of word vector, converts each word into a high-dimensional vector representation, and then extracts features and understands semantics through a multi-layer Transformer structure, in the analysis process, the intent query model identifies the core analysis target in the query, such as the core analysis target in this embodiment is to analyze the reason for the decline of sales, then the analysis entity is extracted based on this, specifically, the time entity "Q3", the regional entity "East China", the index entity "sales decline", and the intent entity "reason"; in some embodiments, the above-mentioned intent query model outputs the extracted results in a structured form, such as in JSON format, specifically represented as {time: Q3, region: East China, index: sales decline, intent: reason}, thus, it is convenient for the subsequent data processing process to be further processed.

[0029] In this embodiment, for the extracted analysis entity, a corresponding entity search and matching is performed in the knowledge base content to obtain corresponding analysis metadata. Specifically, a connection is established between the intent query model and the knowledge base content, wherein the knowledge base content includes an internal database and external Internet data, and the internal database is used to store data source detailed information of each analysis entity, i.e., analysis metadata, specifically including a name of a data table of the analysis entity, a field name, a data type, a storage location, a data owner, an access permission, and the like. For the analysis entity in the above embodiment, the corresponding analysis metadata is a data table related to the "sales" index entity, and the data related to the "Q3" time entity and the "East China region" region entity is obtained from the data table. In a specific application, a search and matching is performed in the knowledge base content to filter out corresponding data tables, such as a customer order table and a sales record table, which store detailed data related to sales. Meanwhile, a data query is performed from the data table to extract corresponding time entities and region entities, such as using SQL query code to extract data.

[0030] In this embodiment, the rule engine identifies an entity keyword of an intent entity from the analysis entity, such as the entity keyword of the "decline" intent entity in the above "Q3 East China region sales decline reason" embodiment, and based on the entity keyword, the rule engine performs keyword matching from a preset rule matching library to determine a corresponding analysis intent, such as matching the intent entity "decline" to the "attribution analysis" analysis intent, and synchronously triggering a corresponding action rule, i.e., in subsequent research and analysis, in-depth research is performed based on the analysis intent.

[0031] In some embodiments, the analysis intent includes, in addition to the attribution analysis, trend analysis, comparative analysis, distribution analysis, correlation analysis, clustering analysis, etc., wherein the trend analysis mainly studies the law and trend of data change over time, by observing the numerical change of data at different time points, to judge whether it is rising, falling or stable, so as to predict the future development direction, and the specific intent entity keywords for determining the analysis intent are “trend” “trend” “change” etc., when the intent entity extracted in the intent query model includes the above keywords, it can be determined that the analysis intent of the analysis requirement exists trend analysis, and the comparative analysis is to compare two or more comparable data to find out their differences and similarities, in order to evaluate the advantages and disadvantages, changes, etc. between different objects, when the intent entity extracted in the intent query model includes the keywords such as “comparison” “comparison” “and……compared with” etc., it can be determined that the analysis intent of the analysis requirement exists comparative analysis; the distribution analysis is used to study the distribution characteristics of data, including the central tendency of data (such as mean, median, mode), dispersion degree (such as variance, standard deviation) and distribution form of data (such as normal distribution, skewness distribution, etc.), in order to understand the distribution of data in different value range, when the analysis requirement of the user contains the distribution of the concerned data, it can be determined that the analysis intent of the analysis requirement exists distribution analysis, for example, the user inputs “the distribution of the age of the company employees”, then the intent query model determines the existence of the distribution analysis of the analysis intent based on the entity keyword “distribution situation”, for correlation analysis, it aims to find the correlation between different variables in the data set, that is, when one variable changes, whether another variable will also change and how much the change is, through correlation analysis, the hidden rules and patterns in the data can be found, providing basis for decision-making, when the user's analysis requirement involves the relationship between multiple variables, correlation analysis is triggered, for example, the user inputs “what is the relationship between product sales and advertising investment”, then the intent query model identifies this inquiry about variable relationship and performs corresponding rule engine matching to determine the analysis type as correlation analysis, and the clustering analysis is to group the objects in the data set according to their similarity, so that the objects in the same group have high similarity, while the objects between different groups have great difference, through clustering analysis, the data can be divided into different categories, helping users to find the potential structure and pattern in the data, when the user wants to classify or group the data, clustering analysis is triggered, for example, the user inputs “classify the customers of the company according to the consumption behavior”, then the intent query model identifies this classification requirement and performs corresponding rule engine matching to determine the analysis type as clustering analysis.

[0032] It can be understood that in an analysis requirement, multiple analysis intentions can be determined through the cooperation of the intention query model and the rule engine, that is, the attribution analysis and the trend analysis can exist simultaneously, or the attribution analysis and the distribution analysis can exist simultaneously, which is determined based on the analysis requirement of the user. In the embodiment, the multiple analysis intentions and the analysis entity and the analysis metadata generated in the foregoing are stored in a structured form, thereby facilitating the in-depth research on the analysis requirement, and further ensuring the reliability of the data analysis report finally generated.

[0033] S200, planning a preliminary layout of the analysis report based on the preliminary research requirement; In the application, the research requirement of the user is allocated to a theme direction, thereby planning a preliminary layout of the analysis report, that is, determining the research direction of the analysis report, thereby objectively researching and analyzing the data.

[0034] In other embodiments, the step S200 includes: Based on the analysis entity, an entity graph is constructed, and the correlation strength between different entities is calculated; Based on the correlation strength, the analysis requirement is classified, and multiple analysis themes are determined; Based on the analysis theme, the analysis report framework is determined in combination with the analysis intention, and the preliminary layout is generated.

[0035] In the embodiment, the multiple analysis entities in the preliminary research requirement obtained in the step S100 are taken as nodes to construct the entity graph. Specifically, “Q3” is taken as a time node, “East China” is taken as a regional node, “sales decline” is taken as an index node, and “reason” is taken as an intention node. Then, the correlation relationship between different nodes is represented by a directed edge, and the entity graph is constructed. In actual application, the index node is usually taken as the core, and the correlation strength between other nodes and the index node is calculated. In other embodiments, the intention node can also be taken as the core, and the correlation strength between other nodes is calculated. The correlation strength is adaptively adjusted according to the analysis requirement of the user, and is used to measure the closeness of the relationship between different entities. When the analysis theme is determined, the correlation strength between different entities is combined with the theme direction to determine the corresponding analysis theme.

[0036] In some embodiments, the theme direction includes market trend, technology driving, and policy influence. The above theme directions are taken as classification directions to determine multiple analysis themes, which are used for classifying the research tasks of the analysis requirement.

[0037] Specifically, taking the above "Q3 East China region sales decline reason" as an example, in the calculation of the correlation strength, the sales data of the past 5 years and 20 quarters is taken as the data basis to construct the correlation matrix, and the possible reasons for affecting sales in each quarter are recorded, including market activities, technical changes, policy adjustments, etc., and the regional information involved, such as the East China region, the North China region, etc. Taking the index node as the core, the entity pairs for which the correlation strength needs to be calculated are determined, which are "East China region-sales decline" and "Q3-sales decline" in this embodiment. Then the proportion of the number of times that the above two entity pairs appear simultaneously in the data to the total number of records is calculated. For example, the co-occurrence frequency of "East China region" and "sales decline" is calculated, that is, the proportion of the number of quarters in which the sales of the East China region declined in the past 20 quarters to the total number of quarters. The commonality score is calculated. In addition, the correlation degree between the above entity pairs is scored by domain experts, with a score range of 0-10, 0 indicating no correlation and 10 indicating very close correlation. The expert scores of each entity pair are summarized to calculate the average score as the expert score of the entity pair. Then, according to the importance and reliability of the commonality score and the expert score, the corresponding weights are allocated, and the correlation strength is calculated by using the weighted average method. In actual application, the weights are adjusted according to the specific circumstances to ensure the accuracy and reasonableness of the calculation results of the correlation strength. In addition, for different types of entity pairs, different calculation methods and weight allocation strategies can be used as needed, and the method is not limited to the above commonality score or expert score method.

[0038] In this embodiment, according to the business needs and analysis intentions, a correlation strength threshold is preset, and based on the preset threshold, the correlation strength between different entity pairs is determined. Specifically, if the correlation strength is higher than the correlation strength threshold, it indicates that the corresponding entity pair has strong correlation. The entity pairs with strong correlation are combined with the theme direction to determine the corresponding analysis theme. For example, in one possible embodiment, if the correlation strength between the "East China region-sales decline" entity pair is 0.6, which is higher than the correlation strength threshold, it indicates that the entity pair has strong correlation. Further, based on market trends, technical driving and policy influence, the analysis theme is determined. Specifically, the analysis theme can be determined as "East China region market trend and sales decline correlation analysis", that is, in the subsequent research process, the specific performance of the East China region market trend, such as market demand increase, consumer preference change, etc., and the contribution degree of the corresponding specific performance to the sales decline can be further studied, the driving factors of the market trend change can be analyzed, and the future market trend and sales decline trend of the East China region can be predicted. The analysis theme can also be determined as "technology-driven innovation and East China region sales difference analysis", "policy influence and sales fluctuation analysis".

[0039] It can be understood that the subject direction can be adjusted according to the business needs of the user, and the analysis report framework takes the subject direction as the basic framework, and fills the content of the basic framework with the analysis topics determined under each subject direction, while considering the analysis intention, generating a multi-level research framework for each analysis topic, and for different analysis topics under the same subject direction, the order is sorted according to the correlation strength between entities, and finally a preliminary layout is generated. Under each analysis topic, the order is organized according to the multi-level research framework, so that the entire layout is clear and hierarchical.

[0040] In one possible embodiment, the multi-level research framework of the above-mentioned "East China market trend and sales decline correlation analysis" combined with the analysis intention "attribution analysis" includes (1) clear attribution target, used for specific situation explanation of sales decline, (2) identify potential influencing factors, used for macroeconomic situation analysis, that is, research combined with market economy, consumer preference change, population structure change data, (3) analyze the influence degree of each factor, in this research framework, the regression analysis result is displayed, and each factor coefficient is interpreted, (4) determine the main reason, based on the key factor determination and basis in the above research process. Specifically, the above multi-level research framework can be personalized according to the actual needs of the user, combined with the analysis intention, or combined with the analysis intention to match the similarity from multiple research templates, and then determine the corresponding research framework.

[0041] S300, based on the preliminary layout, decomposing the preliminary research requirement into a plurality of research tasks; In this embodiment, based on the research layout, the preliminary research requirement is divided into a plurality of first-level research tasks according to the subject direction, such as the market trend direction, the technology driving direction and the policy influence direction in the above embodiment, and then in each subject direction, further divided according to the analysis topic, wherein one analysis topic corresponds to one research task, that is, in the subsequent data analysis and research process, each analysis topic under the subject direction is taken as a research unit for separate research, in this process, multiple research tasks can be parallel, greatly shortening the overall research period and improving the research efficiency, for example, in the "East China market trend and sales decline correlation analysis", the research of different analysis topics such as market trend change, technology driving, policy and regulation change can be carried out at the same time, which speeds up the speed of reaching a conclusion.

[0042] Meanwhile, each research task has a clear theme direction and analysis theme as a boundary, which can avoid repeated research between different tasks during the research process, which helps to concentrate resources and efforts, carry out in-depth analysis on specific problems, and further mine the information and rules behind the data, improve the pertinence and effectiveness of the research, and help to find the key factors and potential correlations hidden in complex data, thereby improving the accuracy and reliability of the research results. Each research task is carried out according to the established logic and steps, avoiding one-sidedness and randomness of the research.

[0043] S400, for each research task, in combination with the knowledge base content, respectively, in-depth research is carried out to generate an analysis draft; In other embodiments, the above step S400 comprises: Based on the research task, knowledge retrieval is performed from the knowledge base content to obtain a multi-source data set; Based on the multi-source data set, data analysis is performed in combination with the analysis intent; The results of the data analysis are filled into the analysis report framework to generate an analysis draft.

[0044] As described in step S100, the knowledge base content of the present application specifically includes internal databases and external Internet data, wherein the internal database is the data analysis process of each type of report data, which is usually recorded in the internal database of the enterprise, namely analysis metadata, specifically including the data table name, field name, data type, storage location, data owner, access permission, etc. of the analysis entity, and the external Internet data is specifically the data publicly available on the Internet through networking or data crawling, such as policies and regulations, market data, industry dynamic data, technical literature, etc. which provide data support for research tasks in different theme directions. In actual application, during the research, according to the data required in the multi-level research framework of the theme direction and analysis theme to which the research task belongs, corresponding knowledge retrieval is performed from the knowledge base content to obtain a multi-source data set, and on this basis, data analysis of the research task is carried out.

[0045] In this embodiment, data preprocessing is performed on each data source in the multi-source data set. Specifically, text analysis and structured conversion techniques are used to generate a formatted data set. Specifically, for unstructured or semi-structured text data such as web page content, social media posts, etc., text analysis techniques such as word segmentation, part-of-speech tagging, named entity recognition, etc. in natural language processing are used to decompose the text into meaningful units and extract key information. The parsed text data is then converted into structured data. In some embodiments, entities and relationships in the text can be mapped to fields in a database table, or the text information can be converted to JSON, XML, or other structured formats. For example, the information of events, characters, time, etc. in a news report is extracted and stored in the corresponding database table.

[0046] Further, if the formatted data set has missing or abnormal values, interpolation or mean filling methods are used for processing to obtain a complete data set. Further, according to the research task, feature variables related to the task are extracted from the complete data set. For example, if the research task is to predict the market demand of a product, it may be necessary to extract the price, quality, competitor situation, market trend, etc. of the product as feature variables. Feature variable extraction can also be achieved through data query, statistical analysis, feature engineering, etc. For example, SQL queries are used to extract relevant fields from an internal database as feature variables, or new feature variables are extracted by calculating statistical indicators such as mean, variance, correlation coefficient, etc. of the data, or feature engineering techniques are used to convert and combine original features to generate feature variables.

[0047] Further, a feature selection algorithm is used to determine a key feature set. Feature selection algorithms can remove redundant and irrelevant features to improve model performance and efficiency. In some embodiments, feature selection algorithms include chi-square test, recursive feature elimination, and methods such as L1 regularization, etc. The actual data requirements are determined, such as using chi-square test to evaluate the correlation between each feature and the target variable, and selecting features with high correlation as the key feature set; or using a recursive feature elimination algorithm to determine the optimal feature subset by gradually deleting features and evaluating model performance.

[0048] Further, data integration is performed on the key feature set, and a clustering analysis algorithm is used to classify market data, technical literature, and policy texts to obtain classified data sets. Each classified data set is matched with a corresponding subject direction and used for data analysis of a corresponding research task. In some embodiments, data relevance is also extracted from the classified data sets, a correlation rule mining algorithm is used to generate a rule set related to the research task, and an initial data set is further generated from the rule set. Data compression techniques are used to optimize storage of the initial data set. Through the above steps, the multi-source data set is processed into classified data sets and initial data sets related to the research task.

[0049] In the research process, the above classified data sets and initial data sets are used as the basis for data analysis in combination with the content of the multi-level research framework of the analysis theme in the analysis report framework. Taking the above multi-level research framework of the "market trend and sales decline correlation analysis in the East China region" as an example, the processed classified data sets and initial data sets are used as the basis, and the focus is on the specific phenomenon of sales decline. From the relevant data sets in the market data, time series data related to sales is extracted, and in combination with consumer preference changes and population structure changes related data, the specific situation of sales decline is described in detail. At the same time, the change trend of indicators such as GDP growth rate, inflation rate, and interest rate is analyzed, and a regression analysis method is used to analyze the extracted key feature set. For example, sales is taken as the target variable, and consumer preference change related features and population structure change related features are taken as independent variables to construct a regression model. Through the display of the regression analysis results, the coefficients of each factor are interpreted. Based on the analysis results of the influence degree of each factor in the above research process, the key factors leading to sales decline are determined. Considering the relevance, significance, and influence degree of each factor, and combining the actual situation and business logic, the main reasons for the sales decline are determined.

[0050] In this embodiment, the results of data analysis are quantified based on the quantification template. In one possible embodiment, the possible cases of the quantification template are as follows: templates = [f"{insight['factor']} is the main driving factor, with a contribution of {insight['impact'] * 100:.1f}%", f"significant changes in {insight['factor']} are observed ({insight['change']})", f"verified by {insight['method']}, the correlation between {insight['factor']} and the target indicator is {insight['corr']:.2f}"]. It can be understood that the quantification template is one possible embodiment of quantification processing. In actual application, a similarity calculation matching method can be used to select the corresponding quantification template from the quantification template library for matching to achieve the filling of data analysis results and generate an analysis draft based on the multi-level research framework for each research task.

[0051] In some other embodiments, the above step S400 further includes: For multiple research tasks, a task scheduling algorithm is used to allocate thread resources to generate a thread allocation scheme; Obtain the data flow in the research process, and distribute the data flow into multiple data blocks to generate an initial data block set; For each initial data block set, a digital watermark embedding algorithm is used to assign a unique identifier to each data block to generate a watermark data block set; During the research process, the unique identifier of each data block is extracted from the watermark data block set for integrity verification, and the parallel processing of multiple research tasks is performed based on the thread allocation scheme.

[0052] In this embodiment, the type and number of computing operations required for each research task are analyzed, and the data size that each research task needs to process is determined. The hardware configuration of the system is queried to determine the number of available processor cores. A time slice round-robin algorithm is used to perform execution polling of multiple research tasks. Specifically, thread resources are allocated to each task according to a fixed time slice. When the time slice of a task is used up, the execution of the task is paused, and the thread resources are allocated to the next task, and the cycle is repeated to achieve parallel processing of multiple research tasks. In some other embodiments, a short job first algorithm or a first come first served algorithm can also be used for thread resource allocation.

[0053] In this embodiment, the data stream is specifically the classified data set obtained after processing the multi-source data set, and the classified data set and the initial data set classified under the same topic direction are allocated according to the number of analysis topics under the topic direction, that is, the data blocks of the data stream are copied according to the number of research tasks, to generate an initial data block set, wherein each data block is provided with a unique identifier based on the corresponding analysis topic, which is used to identify the corresponding research task of each data block, so as to realize the independence of data processing between different research tasks and prevent data confusion between different research tasks.

[0054] Further, during the research process, the integrity of the corresponding data block also needs to be verified. Specifically, the unique identifier of the data block also includes verification information such as a timestamp, a data source, etc. When performing data analysis in the research task, the unique identifier needs to be matched with the pre-stored legal unique identifier to judge the uniqueness and integrity of the data, prevent the data from being damaged or maliciously tampered with during transmission, thereby ensuring the reliability of the research data and improving the accuracy of the research results. In addition, the unique identifier is not only used for integrity verification, but also can record the source and processing process of the data block. Through the integrity verification process, the complete history of each data block can be traced, including the creation time, modification time, and information such as which research task processed the data block, which can ensure the transparency and repeatability of the research process.

[0055] In this embodiment, the unique identifier can be set by using various technologies, such as the frequency domain embedding method of discrete cosine transform (DCT) embedding and discrete wavelet transform (DWT) embedding in digital watermark technology, or the spatial domain embedding method of least significant bit (LSB) embedding. Taking DCT embedding as an example, the data block is first subjected to DCT transformation, the unique identifier and other information are encoded and embedded in the transformed coefficients, and then inverse DCT transformation is performed to obtain the data block with embedded watermark. Then, each data block with embedded watermark is stored in a specified storage system to form a set of watermark data blocks, and the relevant information of each data block, such as data block number, storage location, and watermark embedding parameters, is recorded for subsequent management and verification.

[0056] S500, globally integrating and correlatively analyzing each analysis draft, and writing a complete data analysis report; The present application provides a unified data transformation and data analysis process from data collection, arrangement, analysis to report writing, improves the data integration and comprehensive analysis capability, and takes the analysis intention as the core to select a suitable core narrative framework for the data analysis report, thereby improving the rationality, readability and understandability of the data analysis report.

[0057] In other embodiments, the globally integrating and correlatively analyzing each analysis draft comprises: For each of the analysis drafts, the analysis drafts are converted into structured information, and an evidence support network is constructed; Multi-source data fusion is performed based on the evidence support network of each of the analysis drafts, and correlation analysis of the analysis theme is performed.

[0058] In this embodiment, taking the above-mentioned "correlation analysis of market trend and sales decline in East China region" as an example, the BiLSTM-CRF model is used to extract entity-level insights, and key facts such as "the overall GDP growth rate in East China region has decreased from 8% last year to 6.5% currently", "the inflation rate has risen to 3%", and "the bank loan interest rate has increased by 0.5 percentage points" are identified. Through the rule engine, the cause-and-effect assertion is identified, and the conclusion assertion is determined as "these changes in macroeconomic indicators may be important reasons for the decline in sales", and the evidence chain is combed: GDP growth slowdown -> weak market demand growth -> enterprise product sales space squeezed -> sales decline; inflation rises -> raw material prices rise -> enterprise profit space compression or product competitiveness affected -> sales decline; interest rate rises -> enterprise financing difficulty increases -> market promotion and R&D investment decreases -> sales decline.

[0059] Further, the key facts are taken as nodes, such as "GDP growth decline", "inflation rise", "interest rate rise", and "sales decline", and the conclusion assertion is also taken as a node, i.e. "macroeconomic indicator changes lead to sales decline", and then according to the evidence chain, an edge is established between "GDP growth decline" and "sales decline", and the relationship is labeled as "market demand growth weak leads to"; an edge is established between "inflation rise" and "sales decline", and the relationship is labeled as "raw material price rise affects profit or competitiveness leads to"; an edge is established between "interest rate rise" and "sales decline", and the relationship is labeled as "financing difficulty increases affects promotion and R&D leads to".

[0060] Further, the node and edge features are initialized, the feature vector of the node contains the type of fact, such as economic indicator, sales result, specific value, etc.; the feature vector of the edge represents the type of edge, such as causal relationship and relationship strength, etc., and the graph neural network is used for training, and the feature vectors of the nodes and edges are updated through the message passing mechanism, so that the network better understands the complex relationships between factors, and the trained graph structure is visualized and displayed, forming an evidence support network for identifying the correlation between each factor and sales decline.

[0061] Further, each analysis draft contains a series of key facts and conclusion assertions based on the above-mentioned transformation process of the analysis draft. For each key fact in each analysis draft, an entity alignment operation is performed. Specifically, for each entity, a matching entity is searched in the preset entity library. If there is no matching entity, the entity is added to the preset entity library as a new entity. In the entity searching process, a semantic similarity calculation method is used for matching. Thus, the standardization of different entities in different analysis drafts is achieved. In some embodiments, the entities in each draft can also be associated with the corresponding entities in the preset entity library and recorded in the entity library list for subsequent database accumulation for semantic similarity calculation, thereby improving the accuracy of matching.

[0062] Meanwhile, for each conclusion assertion in each analysis draft, a relationship fusion operation is performed. According to the entities in the conclusion assertion, an object index is performed from the entity library list, and the corresponding object entity is obtained from the entity library list. Different object entities are added with relationships. For example, the above-mentioned conclusion assertion "the changes in these macroeconomic indicators may be an important reason for the decline in sales" has assertion relationships, GDP growth decline -> sales decline, inflation rate decline -> sales decline, and bank loan increase -> sales decline. Multiple analysis drafts are fused according to the above-mentioned relationship fusion process to obtain a more complex and comprehensive relationship network.

[0063] In some embodiments, during the fusion process, contradiction detection and resolution are also required for the multi-source data fusion and correlation analysis process. These contradictions may come from inconsistencies between different data sources or conflicts between different analysis conclusions. For example, when analyzing the consumer electronics market, two data sources may give different values for the market share of a certain product, or two analysis conclusions may give opposite predictions for the same market phenomenon. In some embodiments, a corresponding conflict resolution function is constructed to handle data-level conflicts, causal relationship conflicts, and conflicts between different analysis insights, ensuring the rationality of the data in the multi-source data fusion process.

[0064] In other embodiments, the writing of the complete data analysis report includes: Based on the analysis intent, a core narrative framework is selected, and a pre-set corpus is combined to add an introduction, write a conclusion, add suggestions, and organize references for the data analysis report. A text optimization algorithm is used to adjust the order of sentences and conjunctions to generate a data analysis report with clear logic.

[0065] In this embodiment, different core narrative frameworks are selected for the analysis report based on different analysis intentions, such as a problem-solving framework for attribution analysis, which generally proposes a problem, analyzes the causes of the problem, and finally proposes a solution; a time sequence framework for trend analysis, which generally describes the events in chronological order; and a comparative analysis framework for comparative analysis, which generally compares different objects or different time periods, and then maps the results of multi-source data fusion to the corresponding data analysis report to generate a complete data analysis report.

[0066] In this embodiment, the narrative logic chain of the analysis report is determined based on the core narrative framework selected by the analysis report, such as the attribution analysis described above. One possible embodiment of the narrative logic chain is "current situation description → key problem disassembly → driving factor identification → prediction model construction → strategy suggestion". In this process, the analysis content of the data analysis report is filled in combination with the preset corpus and the relationship network integrated after the analysis draft, and finally a logical and clear data analysis report is generated.

[0067] Specifically, the preset corpus includes an introduction template, a conclusion template, a suggestion template, and a reference library. The introduction template generally describes the industry background, research value, and methodology overview. The conclusion template generally describes the core findings of the research analysis process, theoretical contributions, and practical significance. The suggestion template generally includes priority strategies, implementation paths, and risk warnings. The reference library records academic literature, industry reports, and authoritative data sources for filling in the reference section. Furthermore, the content of the conclusion section in the data analysis report is generated by filling in the data in the core narrative analysis process based on the conclusion template.

[0068] In this embodiment, for the data analysis report, a text optimization algorithm is used to detect context association to determine the rationality of the logical coherence of the data analysis report. Specifically, a dependency syntax analysis algorithm is used to analyze the logical relationship between sentences to determine the semantic dependency structure. For the semantic dependency structure, the logical weight of each sentence is calculated, and a PageRank algorithm is used to generate a priority ranking of the sentences. According to the priority ranking, if the logical weight of a sentence is higher than a preset threshold, the sentence is retained in the adjusted order to generate an optimized sentence sequence. From the optimized sentence sequence, a set of conjunctions is obtained, and a rule-based replacement algorithm is used to replace inappropriate conjunctions to obtain smooth sentence connections. For the smooth sentence connections, a text generation model is used to fine-tune the sentences to generate a logical and clear data analysis report.

[0069] Referring to Figure 2 The present application also provides a data analysis report generation system, which comprises: The first processing module 201 is configured to acquire an analysis requirement of a user, and perform intention analysis on the analysis requirement to obtain a preliminary research requirement. The second processing module 202 is configured to plan a preliminary layout of an analysis report based on the preliminary research requirement. The third processing module 203 is configured to decompose the preliminary research requirement into a plurality of research tasks based on the preliminary layout. The fourth processing module 204 is configured to, for each research task, perform in-depth research on the basis of knowledge base content to generate an analysis draft. The fifth processing module 205 is configured to perform global integration and correlation analysis on the analysis drafts, and compile a complete data analysis report.

[0070] It can be understood that the contents in the data analysis report generation method embodiment shown in Figure 1 are all applicable to the data analysis report generation system embodiment, the data analysis report generation system embodiment specifically implements the functions same as the data analysis report generation method embodiment shown in Figure 1 , and achieves the same beneficial effects as the data analysis report generation method embodiment shown in Figure 1 .

[0071] It should be noted that the information interaction and execution process between the above systems, since based on the same concept, the specific functions and the technical effects brought by it can be referred to the method embodiment part, and will not be repeated here.

[0072] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the system is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit or module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or software. In addition, the specific name of each functional unit or module is only for convenient distinction, and does not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0073] Please refer to Figure 3As shown, the embodiment of the present application further provides a computer device 3, comprising a memory 302 and a processor 301 and a computer program 303 stored in the memory 302, and when the computer program 303 is executed on the processor 301, the generation method of the data analysis report is realized.

[0074] The computer device 3 can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The computer device 3 can comprise, but is not limited to, the processor 301 and the memory 302. Those skilled in the art can understand that the computer device 3 can comprise more or less components, or some components are combined, or different components, for example, the computer device 3 can further comprise an input / output device, a network access device and the like. Figure 3 The computer device 3 is only an example and does not constitute a limitation on the computer device 3, and the computer device 3 can comprise more or less components, or some components are combined, or different components, for example, the computer device 3 can further comprise an input / output device, a network access device and the like.

[0075] The processor 301 can be a central processing unit (CPU), and the processor 301 can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0076] The memory 302 can be an internal storage unit of the computer device 3 in some embodiments, for example, a hard disk or a memory of the computer device 3. The memory 302 can also be an external storage device of the computer device 3 in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card and the like. Further, the memory 302 can comprise both the internal storage unit and the external storage device of the computer device 3. The memory 302 is used to store an operating system, an application program, a boot loader, data and other programs, for example, program codes of the computer program and the like. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0077] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is run by a processor to realize the method for generating the data analysis report.

[0078] In the embodiment, the integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program is executed by a processor to implement the steps of the above-mentioned various method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / computer equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunications signal and a software distribution medium. For example, a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunications signal.

[0079] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for generating a data analysis report, characterized in that, include: Obtain the user's analytical needs, and perform intent parsing on the analytical needs to obtain preliminary research requirements; Based on the aforementioned preliminary research needs, a preliminary layout for the planning and analysis report is proposed. Based on the aforementioned preliminary layout, the preliminary research requirements are broken down into multiple research tasks; For each research task, in-depth research is conducted based on the knowledge base content, and an analysis draft is generated. Perform global integration and correlation analysis on all the aforementioned analysis drafts, and compile a complete data analysis report.

2. The method as described in claim 1, characterized in that, The process of acquiring user analysis needs and parsing those needs to obtain preliminary research requirements includes: Input the user's analytical needs into the intent query model and extract the analytical entities; The analysis entity is matched using a rule engine to determine the analysis intent; Based on the analyzed entity, search and match are performed in the knowledge base content to obtain analytical metadata; The analytical intent and the analytical metadata are stored in a structured form to generate preliminary research requirements.

3. The method as described in claim 2, characterized in that, The preliminary layout of the analysis report, based on the aforementioned preliminary research needs, includes: Based on the analyzed entities, an entity graph is constructed, and the correlation strength between different entities is calculated. Based on the correlation strength, the analysis requirements are classified to determine multiple analysis topics; Based on the analysis topic and the analysis intent, the framework of the analysis report is determined, and a preliminary layout is generated.

4. The method as described in claim 3, characterized in that, For each research task, in conjunction with the knowledge base content, in-depth research is conducted to generate an analysis draft, including: Based on the research task, knowledge retrieval is performed from the knowledge base content to obtain a multi-source data set; Based on the aforementioned multi-source data set and in conjunction with the analytical intent, data analysis is performed. Fill the analysis report frame with the results of the data analysis to generate an analysis draft.

5. The method as described in claim 3, characterized in that, The process of conducting in-depth research on each research task, incorporating knowledge base content, and generating analysis drafts also includes: For multiple research tasks, a task scheduling algorithm is used to allocate thread resources and generate a thread allocation scheme. Acquire the data stream during the research process, divide the data stream into multiple data blocks, and generate an initial set of data blocks; For each initial set of data blocks, a digital watermarking embedding algorithm is used to assign a unique identifier to each data block, thereby generating a set of watermarked data blocks; During the research process, a unique identifier for each data block was extracted from the watermarked data block set for integrity verification, and multiple research tasks were processed in parallel based on a thread allocation scheme.

6. The method as described in claim 1, characterized in that, The global integration and correlation analysis of the various analysis drafts includes: For each of the aforementioned analysis drafts, the analysis drafts are converted into structured information to construct an evidence support network; Based on the evidence support network of each of the aforementioned analysis drafts, multi-source data fusion is performed, and correlation analysis of the analysis topics is conducted.

7. The method as described in claim 1, characterized in that, The process of writing a complete data analysis report includes: Based on the analytical intent, select the core narrative framework, combine it with a pre-set corpus, and add an introduction, write conclusions, add suggestions, and organize references for the data analysis report. Text optimization algorithms are used to adjust the sentence order and conjunctions to generate logically clear data analysis reports.

8. A data analysis report generation system, characterized in that, include: The first processing module is used to obtain the user's analysis requirements, and to perform intent parsing on the analysis requirements to obtain preliminary research requirements. The second processing module is used to plan the initial layout of the analysis report based on the aforementioned preliminary research requirements. The third processing module is used to decompose the preliminary research requirements into multiple research tasks based on the preliminary layout. The fourth processing module is used to conduct in-depth research on each research task, combining the knowledge base content, and generate analysis drafts. The fifth processing module is used to perform global integration and correlation analysis on the various analysis drafts and to write a complete data analysis report.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images