Method and system for automatically generating report documents from data

Abstract

The present invention provides a method and system for automatically generating a report document from data, comprising: dynamically maintaining an internal data analysis report knowledge base A of an enterprise and an external report resource knowledge base B of the enterprise; processing the knowledge base A and the knowledge base B to form an analysis report material knowledge base C; dynamically maintaining a data analysis report rule pattern base D, and establishing a mapping relationship between the material knowledge base C and the pattern base D; combining a sequence of data tables to be analyzed into a wide table T; generating a series of fact descriptions s1, s2, ..., and obtaining a deduplicated fact description set S; obtaining a final importance score of each fact description in S; obtaining k different subsets S S ; Filter out N facts according to their importance scores s ; Form corresponding facts s s The statement fs; for each subset S S , generate a paragraph from each fs, and then k subsets S s The corresponding paragraphs form a report. The present invention can automatically generate data reports from data, reducing the development engineering workload and text workload.

Description

Technical Field

[0001] The present invention relates to the technical field of information management, and in particular to a method and system for automatically generating a report document from data. Background Art

[0002] As the scale of enterprises continues to expand, the business and processes become more complex, and the difficulty for business personnel to write reports based on data is also increasing. The relevant difficulties include:

[0003] 1. It is difficult to master database table analysis and query technology;

[0004] 2. It is difficult to understand the complete meaning of the data after layers of processing;

[0005] 3. Report writing requires a lot of human labor.

[0006] The invention patent with publication number CN106897389B discloses a device and method for generating a report, including: importing a report template and at least two data sources; the report template contains at least one project; each project in the at least one project is set in a corresponding area in the report template; data corresponding to each project in the at least one project is extracted from the at least two imported data sources respectively; the extracted data corresponding to each project is copied to the area corresponding to each project; and a report in a preset format is output. Summary of the invention

[0007] In view of the defects in the prior art, the present invention provides a method and system for automatically generating a report document from data.

[0008] According to a method and system for automatically generating a report document from data provided by the present invention, the scheme is as follows:

[0009] In a first aspect, a method for automatically generating a report document from data is provided, the method comprising:

[0010] Knowledge base construction steps: Dynamically maintain the enterprise's internal data analysis report knowledge base A and the enterprise's external report resource knowledge base B;

[0011] NLP processing steps: Process various reports in knowledge base A and knowledge base B to form analysis report material knowledge base C;

[0012] Pattern library construction steps: dynamically maintain the data analysis report rule pattern library D, and establish the mapping relationship between the material knowledge base C and the pattern library D;

[0013] Wide table construction steps: for the data table sequence to be analyzed: t1, t2, ..., tn, combine them into a wide table T according to the correlation and primary and foreign key relationships of each table;

[0014] Fact generation step: For table T, a series of fact descriptions s1, s2, ... of table T are generated by automatically generating SQL query statements; the operation on table T is also performed in the subspace of table T, and the deduplicated fact description set S = {s1, s2, ...} is obtained;

[0015] Evaluate facts step: Based on a series of fact descriptions in the set S in the generate facts step, obtain the importance score of each fact description s1, s2,,... in the fact description set S;

[0016] Determine the topic step: Cluster each fact description s1, s2, etc. in the fact description set S according to the fields involved in the fact to obtain k different subsets S S ;

[0017] Select the fact step: for the subset S S According to the importance scores of each fact, N facts are selected. s ;

[0018] Report generation steps: Based on the filtered facts s , forming a corresponding fact s s The statement fs;

[0019] Report integration step: For each subset S S , generate a paragraph from each fs, and then k subsets S S The corresponding paragraphs make up the report.

[0020] Preferably, the pattern library construction step includes: storing the internal enterprise data involved in each report in the material knowledge base C in the pattern library D, including the system, library, table, field and processing method where the data is located. The data processing method stored in the pattern library D changes dynamically with the changes in the internal enterprise system.

[0021] Preferably, the increasing order of importance of the fact description in the fact generation step is: value, proportion, trend, sequence number, aggregation and anomaly.

[0022] Preferably, the fact evaluation step includes: based on the importance in the fact generation step, obtaining the importance score score1 of each fact, and examining the number of times each fact in the fact description set S appears in all subspaces. The higher the number, the higher the score. This part of the score is score2. The final importance score of each fact description in the fact description set S is score=score1+score2, where score1 and score2 need to be normalized, and the value range of the two variables is [0,1].

[0023] Preferably, the step of generating a report specifically includes: s , based on facts s The fields or keywords, sentence similarity, and processing methods in the material knowledge base C are searched for the same facts as s s For similar clauses f1, f2, ..., query the processing methods and fields involved corresponding to f1, f2, ... in the pattern library D, and then s The keywords and values ​​in are used to replace the expressions in f1, f2, ... accordingly to form a series of expression statements fs corresponding to fact s.

[0024] In a second aspect, a system for automatically generating a report document from data is provided, the system comprising:

[0025] Knowledge base construction module: dynamically maintain the enterprise's internal data analysis report knowledge base A and the enterprise's external report resource knowledge base B;

[0026] NLP processing module: processes various reports in knowledge base A and knowledge base B to form analysis report material knowledge base C;

[0027] Pattern library construction module: dynamically maintain the data analysis report rule pattern library D, and establish the mapping relationship between the material knowledge base C and the pattern library D;

[0028] Wide table construction module: for the data table sequence to be analyzed: t1, t2, ..., tn, according to the correlation and primary and foreign key relationships of each table, a wide table T is formed;

[0029] Fact generation module: For table T, a series of fact descriptions s1, s2, ... of table T are generated by automatically generating SQL query statements; the operation on table T is also performed in the subspace of table T, and the deduplicated fact description set S = {s1, s2, ...} is obtained;

[0030] Evaluation fact module: Based on a series of fact descriptions in the set S generated by the fact generation module, the importance score of each fact description s1, s2,,... in the fact description set S is obtained;

[0031] Determine the topic module: Cluster each fact description s1, s2, etc. in the fact description set S according to the fields involved in the fact to obtain k different subsets S S ;

[0032] Select fact module: For subset S S According to the importance scores of each fact, N facts are selected. s ;

[0033] Generate report module: Based on the filtered factss , forming a corresponding fact s s The statement fs;

[0034] Report integration module: For each subset S S , generate a paragraph from each fs, and then k subsets S S The corresponding paragraphs make up the report.

[0035] Preferably, the pattern library construction module includes: the pattern library D stores the internal enterprise data involved in each report in the material knowledge base C, including the system, library, table, field and processing method where the data is located. The data processing method stored in the pattern library D changes dynamically with the changes in the internal enterprise system.

[0036] Preferably, the importance of the fact description in the fact generation module increases in the following order: value, proportion, trend, sequence number, aggregation and anomaly.

[0037] Preferably, the fact evaluation module includes: based on the importance in the fact generation module, obtaining the importance score score1 of each fact, and examining the number of times each fact in the fact description set S appears in all subspaces. The higher the number, the higher the score. This part of the score is score2. The final importance score of each fact description in the fact description set S is score=score1+score2, where score1 and score2 need to be normalized, and the value range of the two variables is [0,1].

[0038] Preferably, the report generation module specifically includes: s , based on facts s The fields or keywords, sentence similarity, and processing methods in the material knowledge base C are searched for the same facts as s s For similar clauses f1, f2, ..., query the processing methods and fields involved corresponding to f1, f2, ... in the pattern library D, and then s The keywords and values ​​in are used to replace the expressions in f1, f2, ... accordingly to form a series of expression statements fs corresponding to fact s.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] By adopting natural language analysis, key element similarity matching and other means, it is used to realize the automated generation process of data analysis reports from data tables, reduce the development engineering workload and text workload, and provide business personnel with a convenient and fast experience of automatically generating report documents based on data. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0042] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0043] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0044] The embodiment of the present invention provides a method for automatically generating a report document from data, referring to Figure 1 As shown, the method specifically comprises the following steps:

[0045] Knowledge base construction steps: Dynamically maintain the enterprise's internal data analysis report knowledge base A and the enterprise's external report resource knowledge base B. Knowledge base A contains relevant data analysis reports, research reports, special reports and other materials in the industry. Knowledge base B contains various research reports, analysis reports, research papers and other materials published on the Internet.

[0046] NLP processing steps: Process various reports in knowledge base A and knowledge base B, mainly including article segmentation, sentence segmentation, word segmentation, keyword extraction, summary generation and other operations to form analysis report material knowledge base C.

[0047] Pattern library construction steps: Dynamically maintain the data analysis report rule pattern library D. And establish the mapping relationship between the material knowledge base C and the pattern library D. The solution is as follows: For the fragments involving data in C, record the processing method of the data in the internal system of the enterprise, including the systems, tables, fields and processing methods involved, and record them in the form of data query statements. D retains the mapping relationship between the data in C and the corresponding query statements.

[0048] Specifically, the pattern library construction steps include: the pattern library D stores the internal enterprise data involved in each report in the material knowledge base C, including the system, library, table, field and processing method where the data is located. The data processing method stored in the pattern library D changes dynamically with the changes in the internal enterprise system.

[0049] Wide table construction steps: for the data table sequence to be analyzed: t1, t2, ..., tn, combine them into a wide table T according to the correlation and primary and foreign key relationships of each table;

[0050] Fact generation step: For table T, a series of fact descriptions s1, s2, ... of table T are generated by automatically generating SQL query statements; the operation of table T is also performed in the subspace of table T, and the deduplicated fact description set S = {s1, s2, ...} is obtained. The importance of the following fact descriptions increases as follows:

[0051] For example, when the value of field a is a1, the value of field b is b1.

[0052] Proportion, for example: the data with value a1 in column a accounts for p;

[0053] Trend, for example: column a is in an upward / downward trend within a certain time period;

[0054] Sequence number, for example: when the value in column a is a1, the value in column b is b1, and the order is k;

[0055] Aggregation, for example: the average of column a is a1, the sum is b1, the variance is c1, and so on;

[0056] Exception, for example: the highest / lowest value in column a is a1.

[0057] The operation on table T is also performed in the subspace of table T. The subspace of table T is a subtable obtained by filtering and screening table T to obtain a deduplicated fact description set S.

[0058] Evaluate the fact step: for a series of fact descriptions in the set S, based on the importance in the fact generation step, obtain the importance score score1 of each fact, and examine the number of occurrences of each fact description in the fact description set S in the entire subspace. The higher the number, the higher the score. This part is scored score2. The final importance score of each fact description in the fact description set S is score = score1 + score2, where score1 and score2 need to be normalized, and the value range of the two variables is [0,1].

[0059] Determine the topic step: Cluster each fact description s1, s2, etc. in the fact description set S according to the fields involved in the fact to obtain k different subsets S s ;

[0060] Select the fact step: for the subset S S According to the importance scores of each fact, N facts are selected. s .

[0061] Report generation steps: Based on the filtered facts S , the filtered facts s S , based on facts SThe fields or keywords, sentence similarity, and processing methods in the material knowledge base C are searched for the same facts as s s For similar clauses f1, f2, ..., query the processing methods and fields involved corresponding to f1, f2, ... in the pattern library D, and then s The keywords and values ​​in are used to replace the expressions in f1, f2, .... accordingly to form a statement fs corresponding to fact s.

[0062] Report integration step: For each subset S S , generate a paragraph from each fs, and then k subsets S S The corresponding paragraphs make up the report.

[0063] The present invention also provides a system for automatically generating a report document from data, the system specifically comprising:

[0064] Knowledge base construction module: Dynamically maintain the enterprise's internal data analysis report knowledge base A and the enterprise's external report resource knowledge base B. Including functions such as building a historical report knowledge base and a rule pattern library. Among them, knowledge base A contains relevant data analysis reports, research reports, special reports and other materials in the industry. Knowledge base B contains various research reports, analysis reports, research papers and other materials published on the Internet.

[0065] NLP processing module: including functions such as NLP processing of reports in the knowledge base. Processing of various reports in knowledge base A and knowledge base B, mainly including segmentation, sentence segmentation, word segmentation, keyword extraction, summary generation, etc., to form analysis report material knowledge base C.

[0066] Pattern library construction module: Dynamically maintain the data analysis report rule pattern library D. And establish the mapping relationship between the material knowledge base C and the pattern library D. The solution is as follows: For the fragments involving data in C, record the processing method of the data in the internal system of the enterprise, including the systems, tables, fields and processing methods involved, and record them in the form of data query statements. D retains the mapping relationship between the data in C and the corresponding query statements.

[0067] Specifically, the pattern library construction module includes: the pattern library D stores the internal enterprise data involved in each report in the material knowledge base C, including the system, library, table, field and processing method where the data is located. The data processing method stored in the pattern library D changes dynamically with the changes in the internal enterprise system.

[0068] Wide table construction module: including related wide table construction, fact generation and evaluation, theme determination and other functions. For the data table sequence to be analyzed: t1, t2, ..., tn, according to the correlation and primary and foreign key relationships of each table, a wide table T is combined.

[0069] Fact generation module: For table T, a series of fact descriptions S0 of table T are generated by automatically generating SQL query statements. The following fact descriptions are in increasing order of importance:

[0070] For example, when the value of field a is a1, the value of field b is b1.

[0071] Ratio, for example: the proportion of data in column a whose value is a1 is p.

[0072] Trend, for example: column a is in an upward / downward trend within a certain time period.

[0073] Sequence number, for example: when the value in column a is a1, the value in column b is b1, and the order is k.

[0074] Aggregation, for example: the average of column a is a1, the sum is b1, the variance is c1, and so on.

[0075] Exception, for example: the highest / lowest value in column a is a1.

[0076] The operation on table T is also performed in the subspace of table T. The subspace of table T is a subtable obtained by filtering and screening table T to obtain a deduplicated fact description set S.

[0077] Evaluate the fact module: for a series of fact descriptions in the set S, based on the importance in the generated fact module, obtain the importance score score1 of each fact, and examine the number of occurrences of each fact description in the fact description set S in the entire subspace. The higher the number, the higher the score. This part is scored score2. The final importance score of each fact description in the fact description set S is score = score1 + score2, where score1 and score2 need to be normalized, and the value range of the two variables is [0,1].

[0078] Determine the topic module: Cluster each fact s in the fact description set S according to the fields involved in the fact to obtain k different subsets S S .

[0079] Select fact module: For subset S s According to the importance scores of each fact, N facts are selected. s .

[0080] Generate report module: filter the facts s , based on facts s The fields or keywords, sentence similarity, and processing methods in the material knowledge base C are searched for the same facts as s sFor similar clauses f1, f2, ..., query the processing methods and fields involved in f1, f2, ... in the pattern library D, and then s The keywords and values ​​in are used to replace the expressions in f1, f2, ... accordingly to form a statement fs corresponding to fact s.

[0081] Report integration module: For each subset S s , generate a paragraph from each fs, and then k subsets S s The corresponding paragraphs make up the report.

[0082] The embodiment of the present invention provides a method and system for automatically generating report documents from data, performing NLP processing on historical data analysis reports to generate a material library; automatically generating various facts based on the data, evaluating the facts, selecting the facts to determine the theme, and combining articles based on the similarity of the themes, thereby achieving the technical problem of automatically generating data reports from data.

[0083] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0084] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for automatically generating a report document from data, characterized in that: include: Knowledge base construction steps: Dynamically maintain the enterprise's internal data analysis report knowledge base A and the enterprise's external report resource knowledge base B; NLP processing steps: Process various reports in the enterprise's internal data analysis report knowledge base A and the enterprise's external report resource knowledge base B to form a material knowledge base C; Pattern library construction steps: dynamically maintain the data analysis report rule pattern library D, and establish a mapping relationship between the material knowledge base C and the data analysis report rule pattern library D; Wide table construction steps: for the data table sequence to be analyzed: t1, t2, ..., tn, combine them into a wide table T according to the correlation and primary and foreign key relationships of the data tables to be analyzed; Fact generation step: for a wide table T, a series of fact descriptions s1, s2, ... of the wide table T are generated by automatically generating SQL query statements; the subspace of the wide table T is a subtable obtained by filtering and screening the wide table T, and the operation of the wide table T is performed in the subspace of the wide table T at the same time, thereby obtaining a deduplicated fact description set S = {s1, s2, ...}; Evaluate facts step: Based on a series of fact descriptions in the fact description set S in the generate facts step, obtain the importance score of each fact description in the fact description set S; Determine the topic step: Cluster each fact description in the fact description set S according to the fields involved in the fact to obtain k different subsets ; Select the fact step: for the subset In each fact, select N facts according to their importance scores ; Steps to generate report: Based on the filtered facts , forming a corresponding fact The statement fs; The report generation step specifically includes: , based on facts The fields or keywords, sentence similarity, and processing methods in the material knowledge base C are searched for facts. For the similar clauses f1, f2, ..., the processing methods and fields involved in the clauses are searched in the data analysis report rule pattern library D, and the corresponding processing methods and fields involved are searched according to the facts. The keywords and values ​​in the clause are replaced accordingly to form a corresponding fact. A series of statements fs; Report integration steps: For each sub-collection , each fs generates a paragraph, and then k subsets The corresponding paragraphs make up the report.

2. The method for automatically generating a report document from data according to claim 1, characterized in that: The pattern library construction steps include: the data analysis report rule pattern library D stores the internal enterprise data involved in each report in the material knowledge base C, including the system, library, table, field and processing method where the data is located. The data processing method stored in the data analysis report rule pattern library D changes dynamically with the changes in the internal enterprise system.

3. The method for automatically generating a report document from data according to claim 1, characterized in that: The increasing order of importance of fact descriptions in the fact generation step is: value, proportion, trend, sequence number, aggregation and anomaly.

4. The method for automatically generating a report document from data according to claim 3, characterized in that: The step of evaluating facts includes: obtaining the importance score score1 of each fact based on the importance in the step of generating facts; and examining the number of occurrences of each fact in the fact description set S in all subspaces, the higher the number, the higher the score, which is represented by score score2; the final importance score of each fact description in the fact description set S is score=score1+score2, wherein score1 and score2 need to be normalized, and the value range of the two variables is [0,1].

5. A system for automatically generating a report document from data, characterized in that: include: Knowledge base construction module: dynamically maintain the enterprise's internal data analysis report knowledge base A and the enterprise's external report resource knowledge base B; NLP processing module: processes various reports in the enterprise's internal data analysis report knowledge base A and the enterprise's external report resource knowledge base B to form a material knowledge base C; Pattern library construction module: dynamically maintain the data analysis report rule pattern library D, and establish the mapping relationship between the material knowledge base C and the data analysis report rule pattern library D; Wide table construction module: for the data table sequence to be analyzed: t1, t2, ..., tn, according to the correlation and primary and foreign key relationships of each table, a wide table T is formed; Fact generation module: For a wide table T, a series of fact descriptions s1, s2, ... of the wide table T are generated by automatically generating SQL query statements; the subspace of the wide table T is a subtable obtained by filtering and screening the wide table T, and the operation of the wide table T is performed in the subspace of the wide table T at the same time, thereby obtaining a deduplicated fact description set S = {s1, s2, ...}; Evaluation fact module: Based on a series of fact descriptions in the fact description set S generated by the fact generation module, obtain the importance score of each fact description in the fact description set S; Determine the topic module: Cluster each fact description in the fact description set S according to the fields involved in the fact to obtain k different subsets ; Select fact module: for subsets In each fact, select N facts according to their importance scores ; Generate report module: Based on the filtered facts , forming a corresponding fact The statement fs; The report generation module specifically includes: , based on facts The fields or keywords, sentence similarity, and processing methods in the material knowledge base C are searched for facts. For the similar clauses f1, f2, ..., the processing methods and fields involved in the clauses are searched in the data analysis report rule pattern library D, and the corresponding processing methods and fields involved are searched according to the facts. The keywords and values ​​in the clause are replaced accordingly to form a corresponding fact. A series of statements fs; Report Integration Module: For each sub-collection , each fs generates a paragraph, and then k subsets The corresponding paragraphs make up the report.

6. The system for automatically generating a report document from data according to claim 5, characterized in that: The pattern library construction module includes: the data analysis report rule pattern library D stores the internal enterprise data involved in each report in the material knowledge base C, including the system, library, table, field and processing method where the data is located. The data processing method stored in the data analysis report rule pattern library D changes dynamically with the changes in the internal enterprise system.

7. The system for automatically generating a report document from data according to claim 5, characterized in that: The importance of the fact description in the fact generation module increases in the following order: value, proportion, trend, sequence number, aggregation and anomaly.

8. The system for automatically generating a report document from data according to claim 7, characterized in that: The fact evaluation module includes: obtaining the importance score score1 of each fact based on the importance in the fact generation module; and examining the number of occurrences of each fact in the fact description set S in all subspaces, the higher the number, the higher the score, represented by score score2; the final importance score of each fact description in the fact description set S is score=score1+score2, where score1 and score2 need to be normalized, and the value range of the two variables is [0,1].

Images