Intelligent number asking system and method

Abstract

The invention discloses an intelligent number asking system and method. By supporting multi-mode instruction collection (including voice, pictures, texts and the like), the flexibility and convenience of questioning of the user are greatly improved. The user can select the most suitable input mode to express the query requirement according to own habits or scene requirements without being limited by a single text input mode. Through the custom SQL generation module, necessary query elements (such as a query entity and a data query type) in a user instruction can be intelligently identified, and an SQL query statement conforming to a user intention is automatically generated by utilizing a pre-constructed knowledge base and a trained text-to-SQL model, so that the complexity of directly compiling the SQL statement by a user is reduced, and the user experience is improved. The query accuracy and efficiency are improved, a full link from a natural language to SQL statements and then to data result output is realized, and business personnel who do not know SQL can query data in a database through common dialogues.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and data query, and in particular to an intelligent number query system and method. Background Art

[0002] In today's data-driven era, data query and analysis have become key links in enterprise decision-making and business optimization. Traditional data query systems are usually divided into two major user groups: data analysts and business personnel. These two types of users access and use data in different ways to meet their respective needs.

[0003] As advanced users of data queries, data analysts usually have solid SQL programming skills. Through professional data platforms, they can handwrite complex SQL query statements to achieve in-depth mining and analysis of data. However, although this method is highly flexible, it also has significant limitations. First, data analysts need to have an in-depth understanding of the structure of the business database and the relationship between tables, which often requires a lot of time and learning costs. Secondly, handwritten SQL query statements are not only inefficient, but also prone to errors, especially when dealing with large-scale data sets, performance issues are particularly prominent. Finally, the display form of query results is mostly tables, which lack intuitiveness and interactivity, and are not conducive to rapid understanding and insight into data.

[0004] Unlike data analysts, business personnel are more focused on querying data through an intuitive and easy-to-use interface to meet daily business needs. The current data query system provides business personnel with a data query function page, allowing them to filter data by selecting preset filter conditions. Although this method is simple and easy to use, it has obvious limitations. On the one hand, the conditional filter items provided by the business management background are limited in scope and it is difficult to meet the diverse query needs of business personnel. Once it is necessary to query data beyond the preset range, it is necessary to rely on the technical team for customized development, which not only increases development costs, but also prolongs query response time. On the other hand, due to the limited understanding of the database structure by business personnel, it is difficult for them to build complex query logic on their own, which limits the depth and breadth of data queries.

[0005] To sum up, the current data query system has obvious shortcomings in meeting users' diverse and efficient query needs. Summary of the invention

[0006] The present application provides an intelligent data query system and method for solving the problem that the current data query system cannot meet the user's diversified and efficient query needs.

[0007] In a first aspect, the present application provides an intelligent number-asking method, the method comprising:

[0008] A multimodal instruction collection module, used to obtain multimodal number-asking instructions from users; wherein the multimodal includes but is not limited to at least two of the following: voice, picture, and text;

[0009] A text instruction determination module, used to determine the instruction text corresponding to the multi-modal numerical instruction;

[0010] A custom structured query language SQL generation module is used to determine the necessary query elements contained in the instruction text; wherein the necessary query elements include a query entity and a data query type; determine the target metadata associated with the query entity and a target SQL query example of the data query type from a pre-built knowledge base; generate a target SQL statement corresponding to the instruction text based on an input prompt statement through a pre-trained text-to-SQL model; wherein the prompt statement includes the instruction text, the target metadata, the target SQL query example and a pre-configured text prompt, and the text prompt is used to indicate the generation of an SQL statement matching the instruction text;

[0011] A processing module, used to query data results in a pre-configured database through the target SQL statement;

[0012] The interactive module is used to output the data results.

[0013] In a second aspect, the present application further provides an intelligent number-asking device, the device comprising:

[0014] Obtaining a multimodal number asking instruction from a user; wherein the multimodal includes but is not limited to at least two of the following: voice, picture, and text;

[0015] Determine necessary query elements contained in the instruction text; wherein the necessary query elements include a query entity and a data query type;

[0016] Determine, from a pre-built knowledge base, target metadata associated with the query entity and a target SQL query example of the data query type;

[0017] Generate a target SQL statement corresponding to the instruction text based on an input prompt statement through a pre-trained text-to-SQL model; wherein the prompt statement includes the instruction text, the target metadata, the target SQL query example, and a pre-configured text prompt, and the text prompt is used to indicate the generation of an SQL statement matching the instruction text;

[0018] Query data results in a pre-configured database through the target SQL statement;

[0019] Output the data result.

[0020] In a third aspect, the present application provides a computer device, the computer device comprising a processor, the processor being configured to implement the steps of the intelligent number-asking method described above when executing a computer program stored in a memory.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which implements the steps of the intelligent number-asking method described above when executed by a processor.

[0022] The beneficial effects of this application are as follows:

[0023] 1. By supporting multi-modal command collection (including voice, pictures, text, etc.), the flexibility and convenience of users' questions are greatly improved. Users can choose the most suitable input method to express their query needs according to their own habits or scene needs, without being restricted to a single text input method.

[0024] 2. Through the custom SQL generation module, it can intelligently identify the necessary query elements in user instructions (such as query entities and data query types), and use the pre-built knowledge base and trained text-to-SQL model to automatically generate SQL query statements that meet user intentions, reducing the complexity of users directly writing SQL statements, and improving the accuracy and efficiency of queries. It realizes the full link from natural language to SQL statements, and then to data result output, so that business personnel who do not understand SQL can also query data in the database through ordinary conversations, breaking through the conditional filtering limitations of traditional business statistics systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 A schematic diagram of the structure of an intelligent number-asking system provided in an embodiment of the present application;

[0027] Figure 2 A schematic diagram of the specific workflow of the intelligent number-asking system provided in the embodiment of the present application;

[0028] Figure 3 A schematic diagram of the process of intelligent number query provided by an embodiment of the present application;

[0029] Figure 4 It is a structural schematic diagram of a computer device provided in an optional embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0031] In order to meet users' diverse and efficient data query needs, the present application provides an intelligent data query system and method.

[0032] Embodiment 1:

[0033] This application provides an intelligent number-asking system. Figure 1 A schematic diagram of the structure of an intelligent number-asking system provided in an embodiment of the present application, the system comprising:

[0034] The multimodal instruction collection module 11 is used to obtain multimodal digital instruction from the user; wherein the multimodal includes but is not limited to at least two of the following: voice, picture, and text;

[0035] A text instruction determination module 12, used to determine the instruction text corresponding to the multi-modal numerical instruction;

[0036] A custom structured query language SQL generation module 13 is used to determine the necessary query elements contained in the instruction text; wherein the necessary query elements include a query entity and a data query type; determine the target metadata associated with the query entity and a target SQL query example of the data query type from a pre-built knowledge base; generate a target SQL statement corresponding to the instruction text based on an input prompt statement through a pre-trained text-to-SQL model; wherein the prompt statement includes the instruction text, the target metadata, the target SQL query example and a pre-configured text prompt, and the text prompt is used to indicate the generation of an SQL statement matching the instruction text;

[0037] The processing module 14 is used to query the data results in the pre-configured database through the target SQL statement;

[0038] The interaction module 15 is used to output the data results.

[0039] The intelligent question-and-answer system provided in this application is intended to solve the problem of insufficient processing capabilities of complex data queries in traditional question-and-answer systems, especially the challenge of users having difficulty quickly and accurately obtaining the required information from the database under changing and complex query requirements. The intelligent question-and-answer system of this application, through deep integration of AI big model technology, can deeply understand the natural language query text input by the user, automatically identify the user's intention, and accurately convert the user's query requirements into corresponding SQL query statements, thereby lowering the technical threshold for data query operations and allowing non-technical users to easily perform complex database query operations. Multiple key modules are integrated in the intelligent question-and-answer system, including a multimodal instruction collection module 11, a text instruction determination module 12, a custom structured query language SQL generation module 13, a processing module 14, and an interaction module 15. Next, the specific functions and working principles of each module are explained:

[0040] 1. Multimodal instruction collection module 11.

[0041] The multimodal instruction collection module 11 is designed to collect query instructions input by users in different modes (such as voice, picture, text). The multimodal instruction collection module 11 provides users with a more flexible, natural and efficient interactive experience by integrating multiple input methods such as voice, picture, and text, and improves the convenience and accuracy of user queries.

[0042] Exemplarily, voice input, as one of the most intuitive natural language interaction methods, allows users to easily issue instructions to the system like daily conversations. A microphone device is integrated in the multimodal instruction collection module 11 to capture the user's voice signal in real time through the microphone device. For example, a user can use a simple voice command to "query last week's sales data", and the intelligent number-asking system can quickly recognize and process this request without the user having to manually enter cumbersome text information. This method not only improves operational efficiency, but also significantly improves the convenience and naturalness of the user experience.

[0043] Exemplarily, image input provides users with a way to intuitively display the query object. In some scenarios, it may be difficult for users to accurately describe the content of the query in words, or they may want to directly point to a specific visual element. At this time, the user can present the query target directly to the intelligent question system by taking a photo or uploading a picture. The multimodal instruction collection module 11 can recognize and process the uploaded picture to meet the user's more diversified query requests. For example, the multimodal instruction collection module 11 obtains a picture uploaded by the user containing specific product information, and understands the user's query request and processes it by identifying the content contained in the picture.

[0044] As another example, although text input is very popular in the digital age, it is still an indispensable part of the multimodal instruction collection module 11. Text input allows users to describe query requirements in a precise and detailed manner, especially when complex logic or specific parameters need to be expressed. In a multimodal interactive environment, text input can be combined with other modalities to form a complete query instruction. For example, while uploading product pictures, users may also need to specify specific query conditions (such as price range, color preference, etc.) through text input.

[0045] It should be noted that the content of the multimodal query command input by the user can be a professional query command or a non-professional query command. In this way, users do not need to learn specific command vocabulary or grammatical structure, but can use natural language or behavioral habits in daily conversations to express their query needs. For example, the user may input voice: "Help me check last month's performance report" or "I want to see the latest product promotion information", these informal voice commands can be accurately recognized and processed by the system.

[0046] 2. Text instruction determination module 12.

[0047] The text instruction determination module 12 is responsible for processing the multimodal input acquired by the multimodal instruction collection module 11 to extract a clear instruction text therefrom.

[0048] For multi-modal question and answer instructions input by voice, the text instruction determination module 12 can call the voice recognition engine to identify the corresponding words and sentences by analyzing the acoustic features in the voice signal with the help of the voice recognition technology of the voice recognition engine, thereby converting human voice into text. Exemplarily, the original voice signal can be processed by denoising, framing, etc. Information that can represent voice features, such as spectrum features, energy features, etc., is extracted from the processed voice signal. Through the pre-trained voice recognition model, the voice features are analyzed to obtain the instruction text.

[0049] For multimodal question and answer instructions input by pictures, the text instruction determination module 12 can call OCR technology and image recognition technology to process the picture. Among them, OCR technology is mainly used to identify text information in pictures, while image recognition technology can help understand non-text content (such as objects, scenes, etc.) in pictures and infer implicit instructions or requirements. Exemplarily, pictures can be scaled, cropped, denoised, etc. to improve the accuracy and efficiency of subsequent recognition processes. OCR technology is used to identify text areas in pictures and convert them into editable text. Image recognition technology (such as convolutional neural network CNN) is used to analyze non-text content in pictures, such as identifying objects, scenes, etc. in pictures, and combining contextual information (such as time, place, etc.) to infer the user's intention. According to the results of text and image recognition recognized by OCR, combined with predefined rules or machine learning models, clear text instructions are generated.

[0050] For a multi-modal question and number instruction inputted in text, the text instruction determination module 12 may directly determine the multi-modal question and number instruction as the instruction text.

[0051] 3. Custom SQL generation module 13.

[0052] The custom SQL generation module 13 is the core of the intelligent query system. It can obtain the instruction text from the text instruction determination module 12, analyze and process the instruction text, and thus obtain a professional target SQL query statement.

[0053] Exemplarily, the custom SQL generation module 13 performs the following steps:

[0054] A. Use natural language processing (NLP) technology to perform word segmentation, part-of-speech tagging, named entity recognition (NER) and other processing on the instruction text to extract necessary query elements from the instruction text.

[0055] The necessary query elements include query entities and data query types. The query entity refers to the specific object or attribute that the user wants to query, such as "sales data", "user information", "time", etc. These query entities usually appear in the form of nouns or noun phrases in the user's query instructions.

[0056] The data query type refers to the specific database operation type that the user wants to perform, such as "query", "aggregate", "sort", etc. The custom SQL generation module 13 determines the user's data query type by analyzing the verbs or verb phrases in the instruction text and the overall context of the instruction text.

[0057] B. Based on the query entities and data query types identified in the previous step, the custom SQL generation module 13 will further perform a search operation in the pre-built knowledge base to obtain metadata and SQL query examples that are closely related to the query requirements.

[0058] In the knowledge base, metadata is an abstract representation of the database structure. It describes in detail the various components in the database, including but not limited to: database name, field name, field notes, foreign key associations, and data enumeration values. The database name is used as the identifier of the data warehouse, such as "sales_data", which directly points to the collection that stores specific business data (such as sales records). In each table of the database, the field name is like a label for the data, providing a clear identity for each column of data in the table. Taking the "sales_data" table as an example, its fields may include "date" (recording the date of the transaction), "amount" (the amount of the transaction), "customer_id" (the unique identifier of the customer involved in the transaction), etc. These fields together constitute the basic framework of the data table. In addition to the field name, adding notes to the field is also an important part of metadata management. The notes can explain in detail the purpose of the field, the data range, special format requirements, etc., provide necessary guidance for data users, and reduce the possibility of misunderstanding and erroneous operations. Each field is assigned a specific data type, such as date, integer, string, etc. These data types define the type and range of data that the field can store. In a complex database system, there is often a close connection between tables. Foreign key relationships are a mechanism used to describe this connection. Foreign keys can easily implement cross-table queries and data consistency checks to meet more complex and diverse business needs. In the "sales_data" table, a foreign key relationship may be established with the "customers" table through the "customer_id" field to query customer details or perform related data analysis.

[0059] In addition to metadata, the knowledge base also stores SQL query examples. The SQL query examples include SQL query statements and corresponding instructions. These SQL query examples are SQL statement templates pre-written by developers based on the actual functions and query requirements of the database. They provide important references and bases for generating specific SQL query statements.

[0060] In one example, the data in the knowledge base is saved in the form of vectors. By converting the data into vector representation, the vector space can be used to measure the similarity and distance between data items, thereby achieving fast and accurate information retrieval and recommendation. On this basis, the custom SQL generation module 13 can vectorize the query schematic and data data type when the query entity and data query type are obtained to obtain the entity vector of the query entity and the type vector of the data query type. By calculating the similarity between the entity vector and each metadata vector in the knowledge base, the custom SQL generation module 13 can identify the target metadata most relevant to the query entity. At the same time, by calculating the similarity between the type vector and the SQL query example vector, the SQL query example that best matches the user's query intent is determined.

[0061] C. When the custom SQL generation module 13 obtains the target metadata and the target SQL query example, the target metadata, the target SQL query example, the instruction text and the pre-configured text prompt can be spliced ​​to obtain a prompt statement that can be input into the text-to-SQL model.

[0062] The text prompt is used to instruct the generation of an SQL statement matching the instruction text.

[0063] D. Input the acquired prompt sentence into the pre-trained text-to-SQL model. The prompt sentence is processed by the text-to-SQL model to obtain the target SQL sentence corresponding to the instruction text.

[0064] In order to enable the text-to-SQL model to accurately understand the instruction text entered by the user in natural language form and convert it into an SQL query statement that can be executed by the database, in this application, SQL statements that can be used in business can be collected as standard SQL statements. Among them, these SQL statements should cover common database operations, such as query (SELECT), insert (INSERT), update (UPDATE) and delete (DELETE), and should include queries of different complexity, such as single-table query, multi-table connection, subquery, use of aggregate functions, etc. Then, for each standard SQL statement collected, a corresponding sample instruction text is generated. Based on the pre-prepared material knowledge base, the sample metadata and sample SQL query examples corresponding to each sample instruction text are determined. For each sample instruction text, a prompt statement sample is determined based on the sample instruction text, the sample metadata associated with the sample instruction text, and the sample SQL query example. The prompt statement samples corresponding to each sample instruction text obtained are saved in the instruction text sample set to train the basic text-to-SQL model through each prompt statement sample. Among them, each prompt statement sample corresponds to a standard SQL statement.

[0065] In a possible implementation, the sample instruction text may be a natural language instruction text written by a professional according to each standard SQL statement.

[0066] In another possible implementation, the sample instruction text can also be expanded on the basis of the artificially generated sample instruction text to increase the diversity of the sample instruction text and reduce the manpower and material resources required to generate the sample instruction text. For example, the natural language processing (NLP) technology is used to replace the words in the artificially generated sample instruction text with synonyms and near-synonyms to generate more natural language instruction variants. For another example, the artificially generated sample instruction text is expanded through a pre-trained approximate text model.

[0067] Input any prompt statement sample into the basic text-to-SQL model. The basic text-to-SQL model may include a deep learning model such as Text2SQL. The expected SQL statement of the above prompt statement sample may be obtained through the basic text-to-SQL model. According to the standard SQL statement and the expected SQL statement, a loss value is determined. According to the loss value, the basic text-to-SQL model is trained to adjust the parameter values ​​of each parameter of the basic text-to-SQL model.

[0068] It should be noted that the basic text-to-SQL model can be an untrained model, or it can be a model that has the ability to convert natural language to SQL after pre-training based on prompt statement samples in different fields and their corresponding standard SQL statements, and then fine-tuning training is performed based on the instruction text sample set collected for specific business.

[0069] The instruction text sample set for training the text-to-SQL model includes a large number of prompt sentence samples. The above operation is performed on each prompt sentence sample. When the preset convergence condition is met, the text-to-SQL model training is completed.

[0070] Among them, the preset convergence condition can be satisfied when the sum of the loss values ​​corresponding to the prompt sentence samples in the instruction text sample set is less than the preset loss value threshold, or the number of iterations for training the basic text-to-SQL model reaches the set maximum number of iterations, etc. In the specific implementation, the setting can be flexibly made and is not specifically limited here.

[0071] As a possible implementation method, when performing model training, the prompt sentence samples in the instruction text sample set can be divided into training samples and test samples. The basic text-to-SQL model is first trained based on the training samples, and then the reliability of the trained text-to-SQL model is verified based on the test samples.

[0072] 4. Processing module 14.

[0073] The processing module 14 can receive the target SQL statement generated by the custom SQL generation module 13, and execute the target SQL statement in the database to retrieve the data required by the user. For example, the processing module 14 will establish a connection with the database (if it has not been established yet), and use an appropriate database interface (such as JDBC, ODBC, ORM framework, etc.) to send the target SQL statement. After receiving the target SQL statement, the database server will parse it, execute the corresponding database operation (such as SELECT, INSERT, UPDATE, DELETE, etc.), and return a result set (for query operations) or an execution status (for non-query operations).

[0074] In a possible implementation, after the processing module 14 obtains the target SQL statement, it can check whether the target SQL statement complies with the pre-configured SQL grammar rules. Only when it is determined that the target SQL statement complies with the pre-configured SQL grammar rules, the target SQL statement is executed to avoid errors during execution.

[0075] Exemplarily, the pre-configured SQL grammar rules include ensuring that all keywords (such as SELECT, INSERT, UPDATE, DELETE, etc.) used in the target SQL statement are supported by the database and are used in the correct position. For example, the SELECT keyword should be located at the beginning of the query statement. Another exemplary embodiment of the pre-configured SQL grammar rules includes verifying whether the overall structure of the target SQL statement complies with the provisions of the SQL grammar. For example, whether it contains necessary clauses, such as FROM, WHERE, etc., and the correct order and nested relationship between clauses.

[0076] It should be noted that the pre-configured SQL syntax rules can be customized according to specific SQL dialects (such as MySQL, PostgreSQL, SQL Server, etc.), because different database systems may have differences in certain syntax details. In addition, organizations or projects may also define additional syntax rules according to their own needs to restrict or standardize the use of SQL statements and improve the readability and maintainability of the code.

[0077] In a possible implementation, when the user inputs a multimodal question instruction, the multimodal instruction collection module 11 will also identify the user's identity and obtain the account information associated with the user. The account information may include a user name, user ID, role, permission level, etc. The account information is sent to the processing module 14 of the intelligent question system. When the processing module 14 obtains the account information, it can query the preset permission database or permission management system to determine whether the account has the permission to execute the target SQL statement. Among them, the permission verification may be based on multiple factors, such as the user's role (such as administrator, ordinary user, etc.), the department or organization to which the user belongs, and the specific SQL statement type (such as query, update, delete, etc.). If the account has the permission to execute the target SQL statement, the processing module 14 can query the data results in the database through the target SQL statement. If the account does not have the corresponding permission, the processing module 14 should refuse to execute the target SQL statement, and return the corresponding error prompt or insufficient permission information to the user through the interactive module 15 of the intelligent question system.

[0078] In a possible implementation, the processing module 14 can also generate a response text based on the data results obtained by the query to help users understand the query results more intuitively and accurately. Specifically, when the processing module 14 obtains the data results, it can use natural language processing technology (NLP) and predefined templates or algorithms to generate human-readable response text based on the data results. For example, the processing module 14 will parse the data results and identify the key information and structure therein, such as field names, data types, values, etc. According to the type and purpose of the query, the processing module 14 will select a suitable template from a predefined template library. Among them, these templates can be designed for different query requirements and used to generate standardized response text formats. The data results obtained by parsing are filled into the selected template to generate a preliminary response text. The generated preliminary response text is post-processed to obtain the final response text. Among them, the post-processing includes removing redundant information, adjusting the sentence structure, adding necessary explanations, etc., to improve the clarity and usefulness of the text.

[0079] 5. Interaction module 15.

[0080] The interaction module 15 is responsible for presenting the data results obtained by the processing module 14 to the user in a user-friendly manner (such as tables, charts, texts, etc.).

[0081] In a possible implementation, when the data results are displayed in a chart, and the output module includes a chart display submodule, after the chart display submodule obtains the data results sent by the processing module 14, it can select a suitable chart type for display according to the type of data in the data results and the purpose of the query. Among them, the chart types include bar charts, line charts, pie charts, scatter plots, heat maps, etc. According to the data in the data results, the content of the chart, such as title, axis labels, legends, data labels, etc., is configured to ensure the integrity and readability of the chart information. Then use the front-end chart library (such as ECharts, Highcharts, Chart.js, etc.) or the back-end chart rendering tool (such as Matplotlib, Seaborn, etc.) to render the chart according to the pre-configured data and chart parameters. The rendered data results are embedded in the user interface, and the user can directly view the data results displayed by the chart through the interface.

[0082] In a possible implementation, if the processing module 14 generates a response text corresponding to the data result, and the output module includes an animation rendering submodule, the animation rendering submodule first receives the response text sent from the processing module 14. Then, the response text is converted into a corresponding audio stream using text-to-speech (TTS) technology. In this process, the voice pitch, speech rate, volume and other parameters can be adjusted to meet specific situations or user preferences. Then, based on the phoneme information of the audio stream (such as syllables, intonation changes, etc.), the animation rendering submodule further renders a video stream containing a picture of the digital person sending the audio stream. The above-generated audio stream and video stream are synchronously synthesized to ensure a perfect match between audio and video. Finally, the synthesized audio and video stream is displayed to the user through a display device corresponding to the intelligent questioning system (such as a computer screen, a mobile phone screen, a VR / AR device, etc.).

[0083] It should be noted that during the display process, user interactive operations can also be supported, such as pausing, playing, adjusting the volume, switching display modes, etc., to enhance the user's viewing experience.

[0084] In one example, when a user wishes to save the data results obtained from a query to a specific location on a local or remote server, a save request is input through the interactive module 15, along with target path information. When the interactive module 15 receives the save request input by the user and specifies the target path in the save request, the processing module 14 will be responsible for executing the save operation to ensure that the data results can be safely stored in the location specified by the user. For example, the processing module 14 will parse the save request received from the interactive module 15 and extract the target path information therein. The target path may be a path of a local file system or a URL of a network storage service. The processing module 14 determines the data results that need to be saved. Then, based on the target path obtained by the analysis, the processing module 14 will save the prepared data results to the specified location.

[0085] In a possible implementation, after the processing module 14 completes the saving operation, the processing module 14 sends a saving result feedback to the interaction module 15, informing the user whether the data is successfully saved and the specific location of the saving. In this way, the user can perform corresponding subsequent operations according to the feedback result.

[0086] The beneficial effects of this application are as follows:

[0087] 1. By supporting multi-modal command collection (including voice, pictures, text, etc.), the flexibility and convenience of users' questions are greatly improved. Users can choose the most suitable input method to express their query needs according to their own habits or scene needs, without being restricted to a single text input method.

[0088] 2. Through the custom SQL generation module 13, it can intelligently identify the necessary query elements in user instructions (such as query entities and data query types), and use the pre-built knowledge base and trained text-to-SQL model to automatically generate SQL query statements that meet user intentions, reducing the complexity of users directly writing SQL statements, and improving the accuracy and efficiency of queries. It realizes the full link from natural language to SQL statements and then to data result output, so that business personnel who do not understand SQL can also query data in the database through ordinary conversations, breaking through the conditional filtering limitations of traditional business statistics systems.

[0089] Embodiment 2:

[0090] Figure 2 A schematic diagram of the specific workflow of the intelligent number-asking system provided in the embodiment of the present application, the process includes:

[0091] S201: The multimodal instruction collection module of the intelligent number-asking system obtains a multimodal number-asking instruction from a user.

[0092] Among them, multimodality includes but is not limited to at least two of the following: voice, picture, and text.

[0093] S202: The multimodal instruction collection module sends the acquired multimodal question-and-answer instruction to the text instruction determination module of the intelligent question-and-answer system.

[0094] S203: The text instruction determination module determines the instruction text corresponding to the multi-modal numerical instruction.

[0095] S204: The text instruction determination module sends the instruction text to the custom SQL generation module of the intelligent question-asking system.

[0096] S205: The custom SQL generation module determines necessary query elements included in the instruction text.

[0097] Among them, the necessary query elements include query entity and data query type.

[0098] S206: The custom SQL generation module determines the target metadata associated with the query entity and the target SQL query example of the data query type from the pre-built knowledge base.

[0099] S207: The custom SQL generation module generates a prompt statement.

[0100] The prompt statement includes an instruction text, target metadata, a target SQL query example, and a pre-configured text prompt, and the text prompt is used to instruct the generation of an SQL statement that matches the instruction text.

[0101] S208: The custom SQL generation module generates a target SQL statement corresponding to the instruction text based on the input prompt statement through a pre-trained text-to-SQL model.

[0102] S209: The custom SQL generation module sends the target SQL statement to the processing module of the intelligent question-answering system.

[0103] S210: The processing module queries the data results in the pre-configured database through the target SQL statement, and generates a response text according to the data results.

[0104] S211: The processing module assembles the data result and the response text into JSON format.

[0105] S212: The processing module sends the assembled data to the interactive module of the intelligent number-asking system.

[0106] S213: The chart display submodule of the interactive module renders the data results in a chart display and displays the rendered data results.

[0107] S214: The animation rendering submodule of the interaction module generates an audio stream corresponding to the response text; renders a video stream containing a picture of the digital human emitting the audio stream according to the phoneme information of the audio stream; synthesizes the audio stream and the video stream to obtain an audio and video stream; and displays the audio and video stream.

[0108] Embodiment 3:

[0109] Based on the same inventive concept, the present application also provides an intelligent number-asking method. Figure 3 A schematic diagram of a process of an intelligent number-asking method provided in an embodiment of the present application, the process comprising:

[0110] S301: Obtain a multimodal number query instruction from a user; wherein the multimodality includes but is not limited to at least two of the following: voice, picture, and text.

[0111] S302: Determine the instruction text corresponding to the multi-modal number instruction.

[0112] S303: Determine necessary query elements contained in the instruction text; wherein the necessary query elements include a query entity and a data query type.

[0113] S304: Determine the target metadata associated with the query entity and the target SQL query example of the data query type from the pre-built knowledge base.

[0114] S305: Generate a target SQL statement corresponding to the instruction text based on the input prompt statement through a pre-trained text-to-SQL model; wherein the prompt statement includes the instruction text, the target metadata, the target SQL query example, and a pre-configured text prompt, and the text prompt is used to indicate the generation of an SQL statement matching the instruction text.

[0115] S306: querying data results in a pre-configured database through the target SQL statement.

[0116] S307: Output the data result.

[0117] The intelligent number-asking method provided in the present application is applied to a computer device, which may be an intelligent terminal, such as a computer, a robot, etc., or a server, such as an application server, a business server, etc.

[0118] In some possible implementations, determining, from a pre-built knowledge base, target metadata associated with the query entity and a target SQL query example of the data query type includes:

[0119] Obtaining an entity vector of the query entity and a type vector of the data query type;

[0120] The target metadata is determined by the similarity between the metadata vector in the knowledge base and the entity vector, and the target SQL query example is determined by the similarity between the SQL query example vector in the knowledge base and the entity vector.

[0121] In some possible implementations, the text-to-SQL model is trained in the following manner:

[0122] Obtain any prompt statement sample in the instruction text sample set and its corresponding standard SQL statement; wherein the prompt statement sample carries a sample instruction text determined based on the standard SQL statement, sample metadata associated with the sample instruction text, and a sample SQL query example;

[0123] Determine the expected SQL statement based on the prompt statement sample through the basic text to SQL model;

[0124] Based on the expected SQL statement and the standard SQL statement, the basic text-to-SQL model is trained to obtain a trained text-to-SQL model.

[0125] In some possible implementations, after generating a target SQL statement corresponding to the instruction text based on an input prompt statement using a pre-trained text-to-SQL model, and before querying data results in a pre-configured database using the target SQL statement, the method further includes:

[0126] It is determined that the target SQL statement satisfies a pre-configured SQL grammar rule.

[0127] In some possible implementations, after generating a target SQL statement corresponding to the instruction text based on an input prompt statement by using a pre-trained text-to-SQL model, and before querying data results in a pre-configured database by using the target SQL statement, the method further includes:

[0128] Obtaining account information of the account to which the multimodal number query instruction belongs;

[0129] Determine whether the account information has permission to execute the target SQL statement.

[0130] In some possible implementations, the outputting the data result includes:

[0131] Rendering and displaying the data results in a chart;

[0132] Display the rendered data results.

[0133] In some possible implementations, the outputting the data result includes:

[0134] Obtaining the response text corresponding to the data result;

[0135] Generate an audio stream corresponding to the response text;

[0136] Rendering a video stream including a picture of a digital human emitting the audio stream according to the phoneme information of the audio stream;

[0137] synthesizing the audio stream and the video stream to obtain an audio and video stream;

[0138] The audio and video streams are displayed.

[0139] In some possible implementations, the method further comprises:

[0140] receiving an input save request; wherein the save request carries target path information;

[0141] In response to the save request received by the interaction module, the data result is saved to the location of the target path information.

[0142] It should be noted that the principle of solving the problem by the intelligent number-asking method is similar to the principle of solving the problem by the above-mentioned intelligent number-asking system. For details, please refer to the above-mentioned method embodiment, and no detailed description will be made here.

[0143] Embodiment 4:

[0144] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present application, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0145] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0146] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0147] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0148] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0149] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0150] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0151] Embodiment 5:

[0152] On the basis of the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program executable by a processor is stored. When the program runs on the processor, the processor implements the following steps when executing:

[0153] Obtaining a multimodal number asking instruction from a user; wherein the multimodal includes but is not limited to at least two of the following: voice, picture, and text;

[0154] Determine necessary query elements contained in the instruction text; wherein the necessary query elements include a query entity and a data query type;

[0155] Determine, from a pre-built knowledge base, target metadata associated with the query entity and a target SQL query example of the data query type;

[0156] Generate a target SQL statement corresponding to the instruction text based on an input prompt statement through a pre-trained text-to-SQL model; wherein the prompt statement includes the instruction text, the target metadata, the target SQL query example, and a pre-configured text prompt, and the text prompt is used to indicate the generation of an SQL statement matching the instruction text;

[0157] Query data results in a pre-configured database through the target SQL statement;

[0158] Output the data result.

[0159] Since the principle of solving the problem by the above-mentioned computer-readable storage medium is similar to that of the intelligent number-asking method, the implementation of the above-mentioned computer-readable storage medium can refer to the embodiment of the method, and the repeated parts will not be repeated.

[0160] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An intelligent number-asking system, characterized in that: The system comprises: A multimodal instruction collection module, used to obtain multimodal number-asking instructions from users; wherein the multimodal includes but is not limited to at least two of the following: voice, picture, and text; A text instruction determination module, used to determine the instruction text corresponding to the multi-modal numerical instruction; A custom structured query language SQL generation module is used to determine the necessary query elements contained in the instruction text; wherein the necessary query elements include a query entity and a data query type; determine the target metadata associated with the query entity and a target SQL query example of the data query type from a pre-built knowledge base; generate a target SQL statement corresponding to the instruction text based on an input prompt statement through a pre-trained text-to-SQL model; wherein the prompt statement includes the instruction text, the target metadata, the target SQL query example and a pre-configured text prompt, and the text prompt is used to indicate the generation of an SQL statement matching the instruction text; A processing module, used to query data results in a pre-configured database through the target SQL statement; The interactive module is used to output the data results.

2. The system according to claim 1, characterized in that The custom SQL generation module is specifically used to obtain the entity vector of the query entity and the type vector of the data query type; determine the target metadata through the similarity between the metadata vector in the knowledge base and the entity vector, and determine the target SQL query example through the similarity between the SQL query example vector in the knowledge base and the entity vector.

3. The system according to claim 1, characterized in that The text-to-SQL model is trained in the following way: Obtain any prompt statement sample in the instruction text sample set and its corresponding standard SQL statement; wherein the prompt statement sample carries a sample instruction text determined based on the standard SQL statement, sample metadata associated with the sample instruction text, and a sample SQL query example; Determine the expected SQL statement based on the prompt statement sample through the basic text to SQL model; Based on the expected SQL statement and the standard SQL statement, the basic text-to-SQL model is trained to obtain a trained text-to-SQL model.

4. The system according to claim 1, characterized in that The processing module is further configured to determine whether the target SQL statement satisfies a pre-configured SQL grammar rule before querying data results in a pre-configured database through the target SQL statement.

5. The system according to claim 1, wherein: The processing module is further used to obtain the account information of the account to which the multimodal query instruction belongs before querying the data results in the pre-configured database through the target SQL statement; and determine whether the account information has the authority to execute the target SQL statement.

6. The system according to claim 1, wherein: The output module includes a chart display submodule; The chart display submodule is used to render the data results in a chart display; and display the rendered data results.

7. The system according to claim 1, characterized in that The output module includes an animation rendering submodule; The processing module is further used to generate a response text according to the data result; The animation rendering submodule is used to generate an audio stream corresponding to the response text; According to the phoneme information of the audio stream, a video stream including a picture of a digital human emitting the audio stream is rendered; the audio stream and the video stream are synthesized to obtain an audio-video stream; and the audio-video stream is displayed.

8. The system of claim 1, wherein: The interaction module is further used to receive an input save request; wherein the save request carries target path information; The processing module is further configured to save the data result to the location of the target path information in response to the save request received by the interaction module.

9. An intelligent number-asking method, characterized in that: The method comprises: Obtaining a multimodal number asking instruction from a user; wherein the multimodal includes but is not limited to at least two of the following: voice, picture, and text; Determine the instruction text corresponding to the multi-modal number asking instruction; Determine necessary query elements contained in the instruction text; wherein the necessary query elements include a query entity and a data query type; Determine, from a pre-built knowledge base, target metadata associated with the query entity and a target SQL query example of the data query type; Generate a target SQL statement corresponding to the instruction text based on an input prompt statement through a pre-trained text-to-SQL model; wherein the prompt statement includes the instruction text, the target metadata, the target SQL query example, and a pre-configured text prompt, and the text prompt is used to indicate the generation of an SQL statement matching the instruction text; Query data results in a pre-configured database through the target SQL statement; Output the data result.

10. The method according to claim 9, characterized in that The outputting of the data result comprises: Generate a response text according to the data result; Generate an audio stream corresponding to the response text; Rendering a video stream including a picture of a digital human emitting the audio stream according to the phoneme information of the audio stream; synthesizing the audio stream and the video stream to obtain an audio and video stream; The audio and video streams are displayed.

Images