Data statistical analysis method and device for intelligent question and answer and medium

Abstract

The invention discloses a data statistical analysis method and device for intelligent questions and answers and a medium, and the method comprises the steps: obtaining a to-be-queried natural language, and determining an industry field where the natural language is located; in a domain model database, the industry domain is retrieved, and a target statement generation model is obtained; inputting the natural language into the target statement generation model to obtain an SQL query statement; and pulling query data from a statistical database by executing the SQL query statement, and performing statistical analysis on the query data. Data query is automatically and efficiently completed, and the accuracy of data query and statistical analysis is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data statistical analysis method, device and medium for intelligent question and answer. Background Art

[0002] With the advent of the big data era, the importance of data platforms has become increasingly prominent and has become an indispensable part of the core assets of modern enterprises. For many users with non-technical backgrounds, directly operating complex and professional SQL statements to perform data analysis is undoubtedly an extremely arduous challenge.

[0003] At present, data analysis tools provide relatively intuitive graphical interfaces to a certain extent, but even so, these tools still require users to have certain data analysis capabilities and basic knowledge, which makes it impossible to automatically and efficiently understand users' natural language needs, resulting in complex and inaccurate data query and statistical analysis processes. Summary of the invention

[0004] The embodiments of the present application provide a data statistical analysis method, device and medium for intelligent question and answer, which are used to solve the problem that the natural language needs of users cannot be automatically and efficiently understood, resulting in complex and inaccurate data query and statistical analysis processes.

[0005] The present application embodiment adopts the following technical solutions:

[0006] On the one hand, an embodiment of the present application provides a data statistical analysis method for intelligent question and answer, the method comprising: obtaining a natural language to be queried, and determining the industry field to which the natural language belongs; searching the industry field in a domain model database to obtain a target sentence generation model; inputting the natural language into the target sentence generation model to obtain an SQL query statement; by executing the SQL query statement, pulling query data from a statistical database, and performing statistical analysis on the query data.

[0007] In one example, obtaining the natural language to be queried specifically includes: receiving language data input by a user; when the language data is voice, converting the language data into text to obtain the natural language to be queried; when the language data is text, cleaning the language data to obtain the natural language to be queried.

[0008] In one example, determining the industry field of the natural language specifically includes: segmenting the natural language to obtain multiple keywords of the natural language; matching the multiple keywords in a pre-constructed field mapping relationship table; and determining the industry field with the highest matching degree as the industry field of the natural language.

[0009] In one example, before searching the industry field in the domain model database to obtain the target sentence generation model, the method also includes: obtaining sample natural language and sample SQL query statements of the sample natural language; using the sample natural language as sample input and the sample SQL query statement as a sample label, performing supervised training on a large language model to obtain a general sentence generation model; updating the general sentence generation model according to the domain knowledge base of different industries to obtain the target sentence generation model for each industry field to construct a domain model database.

[0010] In one example, the statistical analysis of the query data specifically includes: determining an analysis expression and an analysis method of the language to be queried; calculating the query data through the analysis expression to obtain an analysis result; and feeding back the analysis result to a user according to the analysis method.

[0011] In one example, after performing statistical analysis on the query data, the method further includes: receiving a statistical analysis result score value fed back by a user; when the statistical analysis result score is lower than a score threshold, outputting the query data to the data set to be optimized of the target sentence generation model; and updating the target sentence generation model within a preset period based on the data set to be optimized.

[0012] In one example, before obtaining the natural language to be queried, the method also includes: obtaining the user's login account; searching for the login account's own historical query records within a preset historical time period; extracting account keywords with a frequency higher than a preset threshold from the own historical query records; searching for industry historical query records within the preset historical time period, and extracting hot search keywords with a frequency higher than a preset threshold from the industry historical query records; and prompting the account keywords and the hot search keywords on the query input interface.

[0013] In one example, the inputting the natural language into the target sentence generation model to obtain an SQL query statement specifically includes: inputting the natural language into the target sentence generation model to generate an initial SQL query statement; the initial SQL query statement includes query fields, filtering conditions and sorting methods; performing format verification on the initial SQL query statement, and when the verification fails, correcting the initial SQL query statement to obtain an SQL query statement.

[0014] On the other hand, an embodiment of the present application provides a data statistical analysis device for intelligent question and answer, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the data statistical analysis methods for intelligent question and answer described above.

[0015] On the other hand, an embodiment of the present application provides a non-volatile computer storage medium for statistical analysis of data for intelligent question and answer, storing computer executable instructions, and the computer executable instructions can execute any of the above-mentioned methods for statistical analysis of data for intelligent question and answer.

[0016] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0017] It is able to implement intelligent question and answer through the sentence generation model, automatically and efficiently convert user natural language requirements into SQL query statements, and divide industry fields into different fields. By introducing the corresponding field knowledge base for model optimization, it can personalize the sentence generation models for different industry fields, more accurately understand user query intentions, and improve the accuracy of query statement output. It can automatically, efficiently and accurately understand user natural language requirements, and then convert them into data query statements. It can also improve the accuracy of data query and data statistical analysis, especially for scenarios with excessive and complex query data volumes, with particularly outstanding high performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the present application, some embodiments of the present application will be described in detail below in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a data statistical analysis method for intelligent question and answer provided in an embodiment of the present application;

[0020] Figure 2 An architecture diagram of a data statistical analysis system for intelligent question and answer provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of the structure of a data statistical analysis device for intelligent question and answer provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0023] Some embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 A flowchart of a data statistical analysis method for intelligent question and answer provided in an embodiment of the present application. The method can be applied to different business fields, and some input parameters or intermediate results in the process allow manual intervention and adjustment to help improve accuracy.

[0025] The analysis method involved in the embodiments of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For the convenience of understanding and description, the following embodiments are described in detail by taking the server of the control center as an example.

[0026] It should be noted that the server may be a single device or a system consisting of multiple devices, that is, a distributed server, and this application does not make any specific limitation on this.

[0027] Figure 1 The process in includes the following steps:

[0028] S101: Acquire a natural language to be queried, and determine the industry field to which the natural language belongs.

[0029] In some embodiments of the present application, an interface is provided for users to input natural language query requirements, supporting multiple methods such as text input and voice input.

[0030] Based on this, first, the language data input by the user is received. Then, when the language data is speech, the language data is converted into text to obtain the natural language to be queried. In addition, when the language data is text, the language data is cleaned to obtain the natural language to be queried. It should be noted that the data cleaning process, for example, removes punctuation marks, stop words, etc.

[0031] Furthermore, the natural language is segmented to obtain multiple keywords of the natural language. In the pre-built domain mapping relationship table, multiple keywords are matched. Finally, the industry field with the highest matching degree is determined as the industry field of the natural language. For example, the industry fields include finance, sales, and human resources.

[0032] It should be noted that the field mapping relationship table includes a plurality of keywords corresponding to each industry field, that is, a mapping relationship between each industry field and a plurality of keywords.

[0033] In some embodiments of the present application, since the user is in a certain industry and usually frequently queries certain relevant data within the industry, based on the user's historical behavior data, keywords can be automatically recommended for the user to input, thereby improving the efficiency of natural language input and helping the user to understand the latest dynamic information within the industry.

[0034] Based on this, the user's login account is obtained. Then, the login account's own historical query records within the preset historical length are searched. From the own historical query records, account keywords with a frequency higher than the preset threshold are extracted. Then, the industry historical query records within the preset historical length are searched, and from the industry historical query records, hot search keywords with a frequency higher than the preset threshold are extracted. Finally, the account keywords and hot search keywords are prompted on the query input interface.

[0035] It should be noted that each user will have an industry identification in his or her account information.

[0036] In addition, the user interface can be built into a comprehensive query, book, video, audio, picture, encyclopedia and other query functions, and can also provide simplified and traditional Chinese conversion, synonym replacement, pinyin search, word segmentation search and recommended search capabilities.

[0037] S102: Search the industry domain in the domain model database to obtain a target sentence generation model.

[0038] In some embodiments of the present application, it is necessary to pre-build a domain model database, which includes statement generation models for different industry fields.

[0039] Based on this, first, sample natural language and sample SQL query statements of the sample natural language are obtained. Then, the sample natural language is used as sample input, and the sample SQL query statement is used as a sample label to perform supervised training on the large language model to obtain a general sentence generation model. Finally, according to the domain knowledge base of different industries, the general sentence generation model is updated to obtain the target sentence generation model of each industry field to build a domain model database.

[0040] In other words, a large amount of user query data and corresponding SQL statements are collected and annotated as training data. Advanced training algorithms are used to train large models to ensure that the model can accurately understand user query intent and generate high-quality SQL statements.

[0041] In addition, a domain knowledge base is introduced to optimize models for specific industries (such as finance, sales, human resources, etc.) to improve query accuracy.

[0042] S103: Input the natural language into the target sentence generation model to obtain an SQL query sentence.

[0043] In some embodiments of the present application, natural language is input into the target sentence generation model to generate an initial SQL query statement. The initial SQL query statement includes query fields, screening conditions, and sorting methods. Finally, the initial SQL query statement is formatted and, when the formatting fails, the initial SQL query statement is corrected to obtain an SQL query statement.

[0044] It should be noted that the model determines the user's data needs, such as query fields, filtering conditions, and sorting methods, by parsing natural language keywords, phrases, and contextual information, and generates query statements that comply with SQL syntax specifications.

[0045] It should be noted that the system will also perform necessary permission verification and data security checks on SQL query statements.

[0046] S104: extracting query data from a statistical database by executing the SQL query statement, and performing statistical analysis on the query data.

[0047] Among them, the system interacts with the data middle platform and performs data query operations to pull data from the data middle platform.

[0048] In some embodiments of the present application, first, the analysis expression and analysis method of the language to be queried are determined. Then, the query data is calculated by the analysis expression to obtain the analysis result. Finally, the analysis result is fed back to the user according to the analysis method.

[0049] It should be noted that simple statistical analysis is performed on the query results, such as analysis expressions including sum, average, maximum, minimum, count, etc., and custom statistical functions and analysis methods are supported. For example, the statistical analysis results are presented to users in the form of charts, tables, etc.

[0050] It should be noted that when the user enters natural language, the analysis expression and analysis result of this query will be selected synchronously on the input interface, and the display style of the query template can also be customized.

[0051] In addition, to help users better understand the data, it can also intelligently recommend relevant data insights, trend forecasts, and decision-making suggestions based on statistical analysis results and users' historical query behavior data. For example, the purchase quantity trend of car A across the country.

[0052] In some embodiments of the present application, user feedback is collected in real time, including evaluations on the accuracy of query results, system response speed, and user interface friendliness. Then, the sentence generation model is retrained and optimized based on the feedback, thereby continuously optimizing the query accuracy and efficiency of the large model.

[0053] Based on this, the system will set a scoring threshold to determine whether the analysis results meet the user's expectations and receive the statistical analysis result score from the user. When the statistical analysis result score is lower than the scoring threshold, the system will add the original natural language of the query, the generated SQL query statement, and the query data pulled from the statistical database to the target statement generation model's to-be-optimized data set. The system will periodically (such as weekly, monthly, etc.) check the to-be-optimized data set and use this data to train or adjust the target statement generation model.

[0054] pass Figure 1 The method can realize intelligent question and answer through the sentence generation model, automatically and efficiently convert the user's natural language needs into SQL query statements, and divide the industry field into different fields. By introducing the corresponding field knowledge base for model optimization, the sentence generation model of different industry fields is customized to more accurately understand the user's query intention and improve the accuracy of query statement output, thereby improving the accuracy of data query, especially for the scenario with too much and too complex query data, especially highlighting high performance.

[0055] In addition, the technical threshold is lowered: ordinary users can perform data query and statistical analysis without mastering the SQL language, which lowers the technical threshold for data analysis.

[0056] Improve efficiency: Automatically process user query requests, reduce manual intervention, and improve the efficiency of data processing and statistical analysis.

[0057] Enhanced user experience: Using natural language as the interactive method, it provides intuitive query result display, which enhances the user experience.

[0058] Flexibility and personalization: Supports user-defined query templates and statistical analysis rules to meet the diverse needs of users.

[0059] More intuitively, Figure 2 An architectural diagram of a data statistical analysis system for intelligent question and answer provided in an embodiment of the present application.

[0060] exist Figure 2In the big model training module, the general model is output to the big model processing module, and the big model processing module pulls industry data from the domain knowledge base for big model domain training. The natural language is received through the user interaction interface, and the natural language is input into the big model processing module. The big model processing module outputs SQL statements in the SQL generation module, and the SQL execution engine runs the SQL statements, pulls data from the data center, outputs the query data to the data statistical analysis module, and outputs the statistical analysis results to the result display module. The result display module displays the analysis in the user interaction interface. The user feedback collection module optimizes the big model in the big model training module based on the user feedback results.

[0061] pass Figure 2 The system integrates a user-friendly interactive interface, an advanced large model processing module, an efficient SQL execution engine, a flexible data statistical analysis module, and an intuitive result display module. Users input query requirements through natural language, and the system can automatically convert them into SQL query statements and execute them, then perform rapid statistical analysis, and intuitively display the results in the form of charts, tables, etc. The system greatly reduces the technical threshold for data analysis, improves data processing efficiency and user experience. It solves the problem that current data analysis tools have high technical requirements and complex operations for users, allowing ordinary users to easily complete data query and statistical analysis.

[0062] Figure 3 A schematic diagram of the structure of a data statistical analysis device for intelligent question and answer provided in an embodiment of the present application includes:

[0063] at least one processor; and,

[0064] a memory communicatively connected to the at least one processor; wherein,

[0065] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the above-mentioned data statistical analysis methods for intelligent question and answer.

[0066] Some embodiments of the present application provide a non-volatile computer storage medium for statistical analysis of data for intelligent question and answer, storing computer executable instructions, wherein the computer executable instructions can execute any of the above-mentioned methods for statistical analysis of data for intelligent question and answer.

[0067] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0068] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0069] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0071] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0073] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0074] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0075] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0076] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0077] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the technical principle of the present application should all fall within the protection scope of the present application.

Claims

1. A data statistical analysis method for intelligent question answering, characterized in that: The method comprises: Obtaining a natural language to be queried, and determining the industry field to which the natural language belongs; In the domain model database, the industry domain is searched to obtain a target sentence generation model; Inputting the natural language into the target sentence generation model to obtain an SQL query sentence; By executing the SQL query statement, query data is pulled from the statistical database, and statistical analysis is performed on the query data.

2. The method according to claim 1, characterized in that The obtaining of the natural language to be queried specifically includes: Receiving language data input by a user; When the language data is speech, converting the language data into text to obtain a natural language to be queried; When the language data is text, data cleaning is performed on the language data to obtain the natural language to be queried.

3. The method according to claim 1, characterized in that Determining the industry field in which the natural language is located specifically includes: Segmenting the natural language to obtain multiple keywords of the natural language; Matching the multiple keywords in a pre-built domain mapping relationship table; The industry field with the highest matching degree is determined as the industry field of the natural language.

4. The method according to claim 1, characterized in that Before searching the industry domain in the domain model database to obtain the target sentence generation model, the method further includes: Obtaining a sample natural language and a sample SQL query statement of the sample natural language; Using the sample natural language as sample input and the sample SQL query statement as sample label, performing supervised training on the large language model to obtain a general sentence generation model; According to the domain knowledge bases of different industries, the general sentence generation model is updated to obtain the target sentence generation model of each industry field to construct a domain model database.

5. The method according to claim 1, characterized in that The statistical analysis of the query data specifically includes: Determine the analysis expression and analysis method of the language to be queried; Calculate the query data using the analysis expression to obtain an analysis result; According to the analysis method, the analysis result is fed back to the user.

6. The method according to claim 1, characterized in that After performing statistical analysis on the query data, the method further includes: Receive the statistical analysis result score of user feedback; When the score of the statistical analysis result is lower than the score threshold, the query data is output to the data set to be optimized of the target sentence generation model; Within a preset period, the target sentence generation model is updated according to the data set to be optimized.

7. The method according to claim 1, characterized in that Before obtaining the natural language to be queried, the method further includes: Get the user's login account; Searching for the login account's own historical query records within a preset historical period; Extracting account keywords with a frequency higher than a preset threshold from the historical query records; Searching for industry historical query records within the preset historical time period, and extracting hot search keywords with a frequency higher than a preset threshold from the industry historical query records; The account keyword and the hot search keyword are prompted on the query input interface.

8. The method according to claim 1, characterized in that The step of inputting the natural language into the target sentence generation model to obtain an SQL query sentence specifically includes: Inputting the natural language into the target sentence generation model to generate an initial SQL query statement; the initial SQL query statement includes query fields, screening conditions and sorting methods; The initial SQL query statement is formatted and, when the formatting fails, the initial SQL query statement is modified to obtain an SQL query statement.

9. A data statistical analysis device for intelligent question and answer, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data statistical analysis method of intelligent question and answer as described in any one of claims 1-8.

10. A non-volatile computer storage medium for statistical analysis of intelligent question and answer data, storing computer executable instructions, characterized in that: The computer executable instructions are capable of executing the data statistical analysis method for intelligent question and answer as described in any one of claims 1 to 8.

Images