Report data extraction method, system and equipment and medium

By generating a number-get type model and a visual report configuration tool, combined with a caching mechanism, the problem of low report data extraction efficiency in game release is solved, and flexible and fast data acquisition and efficient report configuration are achieved.

CN119088835AActive Publication Date: 2024-12-06GUANGZHOU YIHU NETWORK TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411132268.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-12-06
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

In the prior art, the extraction of report data during game release requires technicians to manually write SQL query statements, which makes it time-consuming and labor-intensive, and it is difficult to support rapid analysis of business needs, affecting the efficiency of data analysis.

Method used

By generating metadata information, establishing a number-get type model, designing visual report configuration tools and cache mechanisms, dynamically adjusting the report structure, supporting the number-geting conditions of complex expressions, and improving the flexibility and efficiency of data acquisition.

Benefits of technology

It realizes the need to dynamically adapt to the changing number collection without modifying the code, improves the efficiency and accuracy of report data extraction, reduces the difficulty of users to configure data sources, and meets personalized needs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088835A_ABST
    Figure CN119088835A_ABST
Patent Text Reader

Abstract

The invention is suitable for the technical field of computer application, and provides a report data extraction method which comprises the steps that metadata information is generated according to historical data extraction requirements of a report, a data extraction type model is established based on the metadata information, and the metadata information comprises data extraction conditions, data types and value ranges; generating an interface component based on the access type model, obtaining access conditions through the interface component, and obtaining query data based on the access conditions and access query statements; caching the access condition and the access query statement by adopting a caching mechanism; and designing a visual report configuration tool, wherein the report configuration tool is used for dynamically adjusting the structure of the report. According to the method and the device, by establishing the access type model, the changing access requirements are dynamically adapted under the condition that codes are not modified, and different types of access conditions are flexibly matched on the report, so that the report data is extracted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of computer application technology, and in particular, relates to a method, system, device and medium for extracting report data. Background Art

[0002] As an important information carrier, reports provide an intuitive form of expression for converting data into information, and provide a good basis for decision-making reliably and securely. They are the basis of business intelligence strategies and are usually used in the field of data analysis. For example, in a game release, data analysis is usually supported by reports. However, with the intensified competition in the game market and the explosive growth of player behavior data, conventional data reports cannot support the data extraction and analysis needs required by the business.

[0003] In the prior art, in the process of game distribution, in order to obtain data in the report, the technicians need to have a deep understanding of the business needs and then manually write SQL query statements to extract data from the database. This process not only tests the professional skills of the technicians, but is also time-consuming and laborious due to its highly customized characteristics. In addition, with the expansion of business scale and the surge in data requirements, technicians are often trapped in the heavy work of "data extraction" and find it difficult to extricate themselves from the work of conducting higher-level data analysis and application, which in turn affects the efficiency and effectiveness of the entire data analysis process. Summary of the invention

[0004] The embodiments of the present application provide a report data extraction method, system, device and medium, which can solve one of the above-mentioned prior art problems.

[0005] In a first aspect, an embodiment of the present application provides a method for extracting data from a report, comprising:

[0006] Generate metadata information according to the historical data acquisition requirements of the report, and establish a data acquisition type model based on the metadata information, wherein the metadata information includes data acquisition conditions, data types, and value ranges;

[0007] Based on the data acquisition type model, an interface component is generated, data acquisition conditions are obtained through the interface component, and query data is obtained based on the data acquisition conditions and the data acquisition query statement;

[0008] Adopting a cache mechanism to cache the data retrieval conditions and the data retrieval query statements;

[0009] A visual report configuration tool is designed, wherein the report configuration tool is used to dynamically adjust the structure of the report.

[0010] Furthermore, metadata information is generated according to the historical data acquisition requirements of the report, and a data acquisition type model is established based on the metadata information. The metadata information includes the name, data type and value range of the data acquisition condition, including:

[0011] According to the business requirements of the report, a demand analysis is performed to obtain a data acquisition requirement list, wherein the data acquisition requirement list includes metadata information;

[0012] The structure of the model is defined by using XML Schema, and a structured data acquisition type model is established based on the data acquisition requirement list. The data acquisition type model is used to obtain the data acquisition type.

[0013] Furthermore, the step of establishing a data acquisition type model based on the metadata information further includes:

[0014] The data acquisition type model is expanded so that the data acquisition type model supports data acquisition conditions with complex expressions.

[0015] Furthermore, the interface component includes a condition control and an interface control, the condition control is used to input data acquisition conditions, and the interface control is used to display query data;

[0016] The method of generating an interface component based on the data acquisition type model, obtaining a data acquisition condition through the interface component, and obtaining query data based on the data acquisition condition and the data acquisition query statement includes:

[0017] Determine the control type of the conditional control according to the data type and the data access type predefined in the data access type model, wherein the control type includes a text box, a drop-down list and a date selector;

[0018] Obtaining the data acquisition conditions of the data to be acquired through the conditional control;

[0019] The data retrieval condition is input into a data retrieval query statement to obtain query data, and the query data is displayed in the interface control.

[0020] Furthermore, before the data retrieval condition is input into the data retrieval query statement to obtain the query data, the following steps are included:

[0021] Parsing the data retrieval conditions to obtain parsing information, wherein the parsing information includes query conditions, query fields, sorting fields, and grouping fields;

[0022] Selecting a corresponding query template from a preset template library according to the parsed information, wherein the query template includes a template placeholder corresponding to the parsed information, and the template placeholder includes a condition placeholder, a field placeholder, a sorting placeholder, and a segmentation placeholder;

[0023] According to a preset mapping rule, obtaining a database table corresponding to the parsed information;

[0024] Based on the query template and the database table, a data retrieval query statement is generated.

[0025] Furthermore, the caching mechanism is used to cache the data retrieval conditions and the data retrieval query statements, including:

[0026] Matching the data retrieval condition with the data retrieval query statement, and generating a unique cache key based on the data retrieval query statement and the data retrieval condition;

[0027] Establishing a cache data table for cache keys corresponding to the plurality of data retrieval query statements, wherein the cache data table includes a cache value and a cache time, and the cache value is query data corresponding to the data retrieval condition;

[0028] Obtain a cache key to be checked for each data to be retrieved, and search the cache data table for a cache key corresponding to the cache key to be checked;

[0029] If there is a cache key corresponding to the cache key to be queried in the cache data table, returning the cache value to the interface control;

[0030] Based on the preset cache expiration time and cache elimination strategy, by comparing the cache expiration time with the cache time, determining whether the cache key needs to be deleted from the cache data table;

[0031] By calculating the query times of the cache key, the sorting relationship of the cache key in the cache data table is dynamically adjusted.

[0032] Further, the report configuration tool includes a data source configuration module, and the data source configuration module includes data source options defined based on a data acquisition type model;

[0033] The design visualization report configuration tool, which is used to dynamically adjust the structure of the report, includes:

[0034] Based on the selected data source option, the connection information of the report is obtained by using an interface call method, wherein the connection information includes a database and a file system;

[0035] Obtain the field name and field type of the report by dragging and dropping, and determine the interface component based on the field name;

[0036] Setting the conditional control of the interface component to the field name, and setting the interface control of the interface component to the position where the field value is located, to generate a basic report;

[0037] Based on the input data extraction conditions, the field value is determined and the final report is generated.

[0038] In a second aspect, an embodiment of the present application provides a report data extraction system, including:

[0039] The first processing module is used to generate metadata information according to the historical data acquisition requirements of the report, and establish a data acquisition type model based on the metadata information, wherein the metadata information includes data acquisition conditions, data types and value ranges;

[0040] The second processing module is used to generate an interface component based on the data acquisition type model, obtain the data acquisition condition through the interface component, and obtain the query data based on the data acquisition condition and the data acquisition query statement;

[0041] The third processing module is used to cache the data retrieval condition and the data retrieval query statement by using a cache mechanism;

[0042] The fourth processing module is used to design a visual report configuration tool, and the report configuration tool is used to dynamically adjust the structure of the report.

[0043] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data extraction method for the above-mentioned report when executing the computer program.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, including a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the data extraction method for the above-mentioned report is implemented.

[0045] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0046] A report data extraction method of the present application, by establishing a data acquisition type model, dynamically adapts to changing data acquisition requirements without modifying the code, and flexibly matches different types of data acquisition conditions on the report, thereby extracting report data. In addition, by setting a cache mechanism, the query efficiency of the report is improved. In addition, by setting a visual report configuration tool, the difficulty and complexity of user configuration of the data source are reduced, the efficiency and accuracy of data processing are improved, and the personalized report data extraction requirements of users are met. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 It is a flow chart of a method for extracting data from a report provided by an embodiment of the present invention;

[0049] Figure 2 It is a structural schematic diagram of a report data extraction system provided by an embodiment of the present invention;

[0050] Figure 3 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0052] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0053] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0054] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0055] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0056] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0057] See also Figure 1 As shown, the present invention is a method for extracting data from a report, comprising the following steps:

[0058] S100, generating metadata information according to historical data acquisition requirements of the report, and establishing a data acquisition type model based on the metadata information, wherein the metadata information includes data acquisition conditions, data types, and value ranges;

[0059] In this embodiment, by establishing and applying the data acquisition type model, the structured representation and dynamic generation of data acquisition conditions are realized, and the flexibility and maintainability of data acquisition are improved. According to the changes in business needs, the definition of the data acquisition type model can be easily modified without modifying the code logic of data acquisition, thereby improving the scalability of the system. By combining the data acquisition type model with specific business attributes, the accuracy and business relevance of data acquisition are ensured, and the data acquisition needs in different business scenarios are met.

[0060] In some embodiments, the above step S100 includes:

[0061] According to the business requirements of the report, a demand analysis is performed to obtain a data acquisition requirement list, wherein the data acquisition requirement list includes metadata information;

[0062] The structure of the model is defined by using XML Schema, and a structured data acquisition type model is established based on the data acquisition requirement list. The data acquisition type model is used to obtain the data acquisition type.

[0063] In this embodiment, according to the historical data acquisition requirements of the report, a demand analysis method is used to obtain a data acquisition requirement list, which contains metadata information. By analyzing the metadata information, key business attributes such as data acquisition conditions, data types, and value ranges are obtained. According to the metadata information in the data acquisition requirement list, a data extraction algorithm is used to obtain raw data that meets the data acquisition conditions, data types, and value ranges. By screening and filtering the raw data, a data set that accurately matches the business requirements is obtained.

[0064] In this embodiment, according to the business attributes in the data acquisition requirement list, the structure of the XMLSchema definition model is adopted to establish a structured data acquisition type model, and an XML Schema definition file based on the data acquisition conditions is obtained. Specifically, the corresponding data types and constraints in the XMLSchema are used to define the data types and data acquisition conditions of each control in the data acquisition type model, and the specific attributes and constraints of each data acquisition condition are obtained by parsing the data acquisition requirement list, and mapped to the corresponding elements and attributes in the XML Schema definition file to determine the complete structure of the data acquisition type model.

[0065] In this embodiment, the data acquisition type model is used to obtain the corresponding data acquisition type, and the acquisition method of the data acquisition condition is determined by the acquired data acquisition type, so as to determine the specific data fields and screening rules that need to be extracted from the data source. According to the determined data fields and screening rules, the original data is obtained from the specified data source by using database query statements or API calls.

[0066] In this embodiment, an XML parser is used to parse the XML Schema definition file to obtain the structural information of the data acquisition type model, including element names, attribute names, data types, etc., and obtain the memory representation of the data acquisition type model. According to the memory representation of the data acquisition type model, the specific data acquisition logic and constraint conditions of each data acquisition condition are obtained by recursive traversal, and the logical expression of the data acquisition condition is constructed. By using a logical expression parser, the logical expression of the data acquisition condition is parsed and verified to determine whether the logical relationship between the data acquisition conditions is reasonable, so that the data acquisition type model with the data acquisition condition of the complex expression can be parsed, so as to obtain a legal combination of data acquisition conditions.

[0067] In some of the embodiments, establishing a data acquisition type model based on the metadata information further includes: extending the data acquisition type model so that the data acquisition type model supports data acquisition conditions with complex expressions.

[0068] In this embodiment, according to the business attributes in the data acquisition type model, an expression parsing engine is used to support the use of complex expressions. Specifically, if the business requirements involve mathematical calculations, a mathematical operator library is imported to ensure the correctness of the operations and obtain mathematical operation capabilities; if the business requirements include logical judgments, logical operators are introduced to support logical condition judgments and determine the logical judgment capabilities; if it is necessary to call a built-in function, a built-in function library is obtained to support common business calculations; according to business needs, if a custom function is required, a custom function module is constructed to expand the expression parsing capabilities.

[0069] In this embodiment, the complex expression structure is analyzed by an expression parser to obtain the various business attributes involved in the complex expression; the logical structure of the complex expression is determined by grammatical analysis to obtain the abstract syntax tree of the expression; if multiple operations are involved in the expression, a bracket priority parsing algorithm is used to determine the order of operations to ensure that the operation results are correct. According to the variable reference characteristics in the expression, the variable or parameter reference in the report is parsed to obtain relevant data. If the expression has a nested feature, the nested structure is parsed to determine the logical and computational relationship of the nested expression. The expression is error-handled through the grammar and logic verification module to obtain feedback on the correctness of the expression.

[0070] In this embodiment, the original data acquisition type model is expanded to support data acquisition conditions containing complex expressions, thereby improving the expression ability and application scope of the data acquisition type model.

[0071] S200, generating an interface component based on the data acquisition type model, obtaining a data acquisition condition through the interface component, and obtaining query data based on the data acquisition condition and the data acquisition query statement;

[0072] In some embodiments, the interface component includes a condition control and an interface control, the condition control is used to input data acquisition conditions, and the interface control is used to display query data;

[0073] The above step S200 includes:

[0074] Determine the control type of the conditional control according to the data type and the data access type predefined in the data access type model, wherein the control type includes a text box, a drop-down list and a date selector;

[0075] Obtaining the data acquisition conditions of the data to be acquired through the conditional control;

[0076] The data retrieval condition is input into a data retrieval query statement to obtain query data, and the query data is displayed in the interface control.

[0077] In this embodiment, the control type of the conditional control is determined according to the data type of the data to be retrieved. Specifically, the data type of the data to be retrieved is input into the data retrieval type model, and the control type of the conditional control is determined according to the relationship between the data type and the data retrieval type predefined in the XML Schema definition file. In one embodiment, the data retrieval type model is predefined in the XML Schema definition file as follows: if the data type is text, the data retrieval type is a text type; if the data type is a date, the data retrieval type is a date type; if there is a specific value range in the data type, the data retrieval type is a specified type. Accordingly, when the data retrieval type is a text type, the control type is a text box; when the data retrieval type is a date type, the control type is a date selector; when the data retrieval type is a specified type, the control type is a drop-down list.

[0078] In this embodiment, the conditional control is used to obtain the data acquisition conditions input by the user, and an event monitoring mechanism is adopted to capture the user's operation and input on the conditional control to obtain specific data acquisition condition parameters. The obtained data acquisition conditions are input into the data acquisition query statement, the data acquisition conditions are dynamically spliced ​​into the data acquisition query statement, and the query operation is performed through the database connection to obtain the query data results that meet the conditions from the data source.

[0079] In some embodiments, before the data retrieval condition is input into a data retrieval query statement to obtain query data, the following steps are included:

[0080] Parsing the data retrieval conditions to obtain parsing information, wherein the parsing information includes query conditions, query fields, sorting fields, and grouping fields;

[0081] Selecting a corresponding query template from a preset template library according to the parsed information, wherein the query template includes a template placeholder corresponding to the parsed information, and the template placeholder includes a condition placeholder, a field placeholder, a sorting placeholder, and a segmentation placeholder;

[0082] According to a preset mapping rule, obtaining a database table corresponding to the parsed information;

[0083] Based on the query template and the database table, a data retrieval query statement is generated.

[0084] In this embodiment, lexical analysis and syntax analysis based on a finite state machine are used to obtain parsing information such as query conditions, query fields, sorting fields and grouping fields in the data acquisition conditions. Specifically, a lexical analyzer is used to perform lexical analysis on the input data acquisition conditions, and lexical units such as keywords (such as screening, query), operators (such as +, -, =), identifiers (such as table names, column names, databases), numbers, and string literals in the data acquisition conditions are identified according to predefined rules (such as regular expressions, lexical rule tables, etc.). Then, a syntax analysis technique is used to obtain the grammatical relationship and structural information between lexical units, and a syntax analysis tree of the data to be retrieved is obtained by analyzing and identifying the syntax structure. According to the syntax analysis tree, a depth-first traversal algorithm is used to obtain node information such as query conditions, query fields, sorting fields and grouping fields in the syntax analysis tree, and the query conditions, query fields, sorting fields and grouping fields are obtained by extracting and combining the node information.

[0085] In this embodiment, according to the parsed information, a keyword matching algorithm is used to obtain one or more candidate query templates by searching for query templates matching the keywords in the parsed information in a preset template library. For each candidate query template, a semantic similarity calculation method is used to calculate the semantic similarity score between the parsed information and the query template to obtain the matching degree of each candidate query template with the parsed information. According to the matching scores of the candidate query templates, a sorting algorithm is used to sort the candidate templates in descending order, and the query template with the highest matching degree is obtained as the query template finally selected. By parsing the selected query template, the type of template placeholder contained in the query template is determined, and the above template placeholder is matched with the parsed information.

[0086] In this embodiment, the database table name and field information in the parsing information are obtained according to the preset mapping rules. The corresponding database table is searched in the database by the database table name. The structural information of the database table is obtained, including the table name, field name, field type, etc. The field mapping rules are adopted to match the fields in the parsing information with the fields of the database table. According to the field matching results, it is determined whether the parsing information matches the database table structure. If the match is successful, it is determined that the database table is the database table corresponding to the parsing information. If the match fails, continue to search for other database tables until a matching database table is found. After obtaining the database table corresponding to the parsing information, it is associated with the parsing information for subsequent data insertion or update operations.

[0087] In this embodiment, a data retrieval query statement is generated through the above-mentioned query template containing template placeholders and a database table. The user enters the data retrieval conditions through the conditional control, and finally an executable data retrieval query statement is generated. The query statement is generated and fed back to the interface control, thereby realizing the data retrieval operation of the report.

[0088] S300, using a cache mechanism to cache the data retrieval condition and the data retrieval query statement;

[0089] In this embodiment, a caching mechanism is introduced to cache the data retrieval conditions and their corresponding data retrieval query statements and query results, thereby reducing repeated database query operations, improving the response speed and performance of the report, and designing a reasonable cache update strategy to ensure the consistency of cached data with actual data.

[0090] In some embodiments, the step S300 includes:

[0091] Matching the data retrieval condition with the data retrieval query statement, and generating a unique cache key based on the data retrieval query statement and the data retrieval condition;

[0092] Establishing a cache data table for cache keys corresponding to the plurality of data retrieval query statements, wherein the cache data table includes a cache value and a cache time, and the cache value is query data corresponding to the data retrieval condition;

[0093] Obtain a cache key to be checked for each data to be retrieved, and search the cache data table for a cache key corresponding to the cache key to be checked;

[0094] If there is a cache key corresponding to the cache key to be queried in the cache data table, returning the cache value to the interface control;

[0095] Based on the preset cache expiration time and cache elimination strategy, by comparing the cache expiration time with the cache time, determining whether the cache key needs to be deleted from the cache data table;

[0096] By calculating the query times of the cache key, the sorting relationship of the cache key in the cache data table is dynamically adjusted.

[0097] In this embodiment, according to the corresponding relationship between the data retrieval condition and the data retrieval query statement, a match is performed, and the data retrieval query statement and the data retrieval condition are spliced ​​by string concatenation to obtain a preliminary cache key string. By performing MD5 hash calculation on the preliminary cache key string, a 32-bit hash value is obtained to determine a unique cache key. According to the determined unique cache key, a search is performed in the cache system to determine whether a corresponding cache value exists, and whether the query result has been cached. If the corresponding cache value is found in the cache system through the cache key, the cached query result is directly obtained and the result is fed back to the interface control without performing an actual database query operation. If the corresponding cache value is not found in the cache system through the cache key, the actual query operation is performed in the database using the data retrieval query statement to obtain the query result. The query results obtained through the database query are stored in the cache system according to the unique cache key, so that subsequent identical queries can directly obtain results from the cache, thereby improving query efficiency.

[0098] In this embodiment, according to the data characteristics of the cache value, the cache expiration time is preset, such as considering the data size, data type and query frequency of the cache value, and the cache value and the corresponding cache time are stored in the cache at the same time, and a suitable cache elimination strategy is selected to determine whether the old data needs to be eliminated. Specifically, by comparing the cache expiration time with the cache time, it is determined whether each cache key has expired. The preset cache elimination strategy is adopted, and the expired cache keys are sorted according to the query frequency, last access time and other attributes of the cache key to obtain a list of cache keys to be eliminated. By traversing the list of cache keys to be eliminated, the cache keys are deleted from the cache data table in turn, and the corresponding cache space is released to ensure the validity and availability of the cache data. The capacity and used space size of the current cache data table are obtained, and according to the preset cache capacity threshold, it is determined whether it is necessary to further eliminate the unexpired cache keys to maintain the reasonable use of the cache space. If it is necessary to further eliminate the unexpired cache keys, the unexpired cache keys are sorted according to the cache elimination strategy, and the appropriate cache keys are selected for deletion until the cache space utilization reaches the preset reasonable range. After completing the cache key deletion operation, obtain the updated cache data table to ensure the consistency and integrity of the cache data, and provide efficient and reliable data support for subsequent cache read and write operations.

[0099] S400: Design a visual report configuration tool, where the report configuration tool is used to dynamically adjust the structure of the report.

[0100] In this embodiment, by providing a visual configuration interface and an automated data source loading and connection mechanism, the system reduces the difficulty and complexity of user configuration of data sources and improves the efficiency and accuracy of data processing.

[0101] In some of the embodiments, the report configuration tool includes a data source configuration module, and the data source configuration module includes data source options defined based on a data acquisition type model;

[0102] The above step S400 includes:

[0103] Based on the selected data source option, the connection information of the report is obtained by using an interface call method, wherein the connection information includes a database and a file system;

[0104] Obtain the field name and field type of the report by dragging and dropping, and determine the interface component based on the field name;

[0105] Setting the conditional control of the interface component to the field name, and setting the interface control of the interface component to the position where the field value is located, to generate a basic report;

[0106] Based on the input data extraction conditions, the field value is determined and the final report is generated.

[0107] In this embodiment, by setting up a report configuration tool, the user is provided with functions and options for selecting and configuring a data source, thereby generating a final report structure.

[0108] In this embodiment, when the user enters the data source configuration module, the system will automatically load the available data source options according to the current environment, system configuration or pre-set rules. The above data source options include the configured database connection, file system path, API interface, etc. The system obtains a list of available data sources by reading configuration files, querying databases or accessing system services, and displays it to the user. Through the interface call method, the system obtains the connection information of the database and file system, generates a data source list, and when the user selects or confirms the data source to be used, the system further obtains the detailed connection information of the data source through the interface call method. For the database, the connection information includes the database type (such as MySQL, Oracle, SQL Server, etc.), server address, port number, database name, user name and password, etc. For the file system, the connection information includes file path, access rights, etc. Through the above connection information, the report is connected to the database or file system to provide data support for the generation and extraction of the report. In addition, the report configuration tool provides all the configured data sources of the user, as well as their connection status and basic information.

[0109] In this embodiment, the layout of the report is adjusted by dragging and dropping. Specifically, when the user drags a field to the report design area, the system needs to determine the interface component based on the field name and field type of the field. For example, for a text type field, a text box or label can be used; for a numeric type field, a text box with numeric formatting can be used; for a date type field, a date selector or a text box in a specific format can be used. At the same time, the conditional control of the interface component corresponds to the field name, and the interface control of the interface component corresponds to the field value, thereby generating a basic report. When the user enters the data extraction condition in the conditional control, the query data is filled in the interface control, so that the basic report structure is rendered to generate the final report.

[0110] In one embodiment, it also includes: setting a chart suggestion module, the chart suggestion module is used to generate multiple chart types according to the final report, when the user selects a chart type, the system generates a chart component and embeds it into the report.

[0111] Specifically, when the user selects the corresponding data source option, the system calls the data source management service through the RESTful API interface to obtain the connection information of the database and file system. At the same time, it receives the data source list in JSON format, which contains detailed information such as the data source name, type, connection string, etc. Then, the user adjusts the field name by dragging and dropping. Specifically, the report layout is adjusted through the drag and drop API of HTML5. The system uses the Flexbox-based CSS layout to dynamically adjust the layout model and then generate the final report. In the above process, machine learning algorithms (such as random forests) are used to analyze the characteristics of report data and recommend appropriate chart types. After the user selects the chart type, the system uses the Chart.js library to generate a chart component and embeds the corresponding chart into the report through DOM operations.

[0112] See also Figure 2 As shown, the present invention also provides a report data extraction system, the system comprising:

[0113] The first processing module 201 is used to generate metadata information according to the historical data acquisition requirements of the report, and establish a data acquisition type model based on the metadata information, wherein the metadata information includes data acquisition conditions, data types and value ranges;

[0114] The second processing module 202 is used to generate an interface component based on the data acquisition type model, obtain a data acquisition condition through the interface component, and obtain query data based on the data acquisition condition and the data acquisition query statement;

[0115] The third processing module 203 is used to use a cache mechanism to cache the data retrieval condition and the data retrieval query statement;

[0116] The fourth processing module 204 is used to design a visual report configuration tool, wherein the report configuration tool is used to dynamically adjust the structure of the report.

[0117] It is understandable that if Figure 1 The contents of the data extraction method embodiment of the report shown in the figure are applicable to the data extraction system embodiment of the report. The functions specifically implemented by the data extraction system embodiment of the report are similar to those in the example shown in the figure. Figure 1 The data extraction method embodiment of the report shown is the same as that of the embodiment of the report shown, and the beneficial effects achieved are the same as those of the embodiment of the report shown. Figure 1 The beneficial effects achieved by the data extraction method embodiment of the report shown are also the same.

[0118] It should be noted that the information interaction, execution process and other contents between the above-mentioned systems are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0119] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0120] See also Figure 3 As shown, an embodiment of the present invention further provides a computer device 3, comprising: a memory 302 and a processor 301 and a computer program 303 stored in the memory 302. When the computer program 303 is executed on the processor 301, a report data extraction method as described in any one of the above methods is implemented.

[0121] The computer device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will appreciate that Figure 3It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0122] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0123] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 3. Further, the memory 302 may also include both an internal storage unit of the computer device 3 and an external storage device. The memory 302 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0124] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the report data extraction method described in any one of the above methods is implemented.

[0125] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0126] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for extracting data from a report, characterized in that: include: Generate metadata information according to the historical data acquisition requirements of the report, and establish a data acquisition type model based on the metadata information, wherein the metadata information includes data acquisition conditions, data types, and value ranges; Based on the data acquisition type model, an interface component is generated, data acquisition conditions are obtained through the interface component, and query data is obtained based on the data acquisition conditions and the data acquisition query statement; Adopting a cache mechanism to cache the data retrieval conditions and the data retrieval query statements; A visual report configuration tool is designed, wherein the report configuration tool is used to dynamically adjust the structure of the report.

2. The method according to claim 1, characterized in that The metadata information is generated according to the historical data acquisition requirements of the report, and the data acquisition type model is established based on the metadata information. The metadata information includes the name, data type and value range of the data acquisition condition, including: According to the business requirements of the report, a demand analysis is performed to obtain a data acquisition requirement list, wherein the data acquisition requirement list includes metadata information; The structure of the model is defined by using XML Schema, and a structured data acquisition type model is established based on the data acquisition requirement list. The data acquisition type model is used to obtain the data acquisition type.

3. The method according to claim 2, characterized in that The step of establishing a data acquisition type model based on the metadata information further includes: The data acquisition type model is expanded so that the data acquisition type model supports data acquisition conditions with complex expressions.

4. The method according to claim 1, characterized in that The interface component includes a condition control and an interface control, wherein the condition control is used to input data acquisition conditions, and the interface control is used to display query data; The method of generating an interface component based on the data acquisition type model, obtaining a data acquisition condition through the interface component, and obtaining query data based on the data acquisition condition and the data acquisition query statement includes: Determine the control type of the conditional control according to the data type and the data access type predefined in the data access type model, wherein the control type includes a text box, a drop-down list and a date selector; Obtaining the data acquisition conditions of the data to be acquired through the conditional control; The data retrieval condition is input into a data retrieval query statement to obtain query data, and the query data is displayed in the interface control.

5. The method according to claim 4, characterized in that Before the data retrieval condition is input into the data retrieval query statement to obtain the query data, the following steps are included: Parsing the data retrieval conditions to obtain parsing information, wherein the parsing information includes query conditions, query fields, sorting fields, and grouping fields; Selecting a corresponding query template from a preset template library according to the parsed information, the query template comprising a template placeholder corresponding to the parsed information, the template placeholder comprising a condition placeholder, a field placeholder, a sort placeholder and a segment placeholder; According to a preset mapping rule, obtaining a database table corresponding to the parsed information; Based on the query template and the database table, a data retrieval query statement is generated.

6. The method according to claim 1, characterized in that The adopting of a cache mechanism to cache the data retrieval condition and the data retrieval query statement includes: Matching the data retrieval condition with the data retrieval query statement, and generating a unique cache key based on the data retrieval query statement and the data retrieval condition; Establishing a cache data table for cache keys corresponding to the plurality of data retrieval query statements, wherein the cache data table includes a cache value and a cache time, and the cache value is query data corresponding to the data retrieval condition; Obtain a cache key to be checked for each data to be retrieved, and search the cache data table for a cache key corresponding to the cache key to be checked; If there is a cache key corresponding to the cache key to be queried in the cache data table, returning the cache value to the interface control; Based on the preset cache expiration time and cache elimination strategy, by comparing the cache expiration time with the cache time, determining whether the cache key needs to be deleted from the cache data table; By calculating the query times of the cache key, the sorting relationship of the cache key in the cache data table is dynamically adjusted.

7. The method according to claim 1, characterized in that The report configuration tool includes a data source configuration module, and the data source configuration module includes data source options defined based on a data acquisition type model; The design visualization report configuration tool, which is used to dynamically adjust the structure of the report, includes: Based on the selected data source option, the connection information of the report is obtained by using an interface call method, wherein the connection information includes a database and a file system; Obtain the field name and field type of the report by dragging and dropping, and determine the interface component based on the field name; Setting the conditional control of the interface component to the field name, and setting the interface control of the interface component to the position where the field value is located, to generate a basic report; Based on the input data extraction conditions, the field value is determined and the final report is generated.

8. A report data extraction system, characterized in that: include: The first processing module is used to generate metadata information according to the historical data acquisition requirements of the report, and establish a data acquisition type model based on the metadata information, wherein the metadata information includes data acquisition conditions, data types and value ranges; The second processing module is used to generate an interface component based on the data acquisition type model, obtain the data acquisition condition through the interface component, and obtain the query data based on the data acquisition condition and the data acquisition query statement; The third processing module is used to cache the data retrieval condition and the data retrieval query statement by using a cache mechanism; The fourth processing module is used to design a visual report configuration tool, and the report configuration tool is used to dynamically adjust the structure of the report.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Device and method for realizing data dynamic cache

    CN102024062A

  • Big data report pre-storage and calculation method

    CN104298724A

  • Report query method and device, server and storage medium

    CN110032567A

  • Report generation method and device, equipment and storage medium

    CN112487773A

  • Reverse modeling design method and system of low-code development platform based on federal learning

    CN118193484A