Query method, device and storage medium based on natural language and data knowledge

Abstract

The present application discloses a query method, device and storage medium based on natural language and data knowledge, including: constructing a first database based on collected metadata and corresponding data lineage, and constructing a second database based on collected historical business data; determining multiple data lineages corresponding to the metadata in the first database, and determining multiple joint knowledge vectors based on the metadata and the corresponding data lineages; matching a first query vector with multiple joint knowledge vectors, and determining a target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to a first query text of a terminal device, and the first query text is natural language text information; generating a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; and querying the second database using the first SQL language instruction to obtain and return the corresponding first business data to the terminal device.

Description

Technical Field

[0001] The present application relates to the field of intelligent questioning technology, and in particular to a query method, device and storage medium based on natural language and data knowledge. Background Art

[0002] Big data and artificial intelligence technologies are developing rapidly. In the whole process of data aggregation, governance, management, and application, the data capabilities of many data assets are precipitated from multiple fields inside and outside the enterprise. Users are increasingly in urgent need of finding the data they need for a specific scenario in big data, and the intelligent data query system can help users obtain data simply and conveniently.

[0003] In the prior art, there are specific examples of using metadata to generate knowledge graphs and using the knowledge graphs. However, there are still many defects in using knowledge graphs for data query. For example, since there may be errors and uncertainties in the relationship extraction and entity linking in the knowledge graph, the accuracy of using the knowledge graph generated based on metadata to help users obtain data is low, which in turn affects the quality of the data and the credibility of the analysis results.

[0004] With regard to the technical problem existing in the above-mentioned prior art that there may be errors and uncertainties in the relationship extraction and entity linking in the knowledge graph, the accuracy of the method of using the knowledge graph generated based on metadata to help users obtain data is low, which in turn affects the quality of the data and the credibility of the analysis results, no effective solution has been proposed so far. Summary of the invention

[0005] The embodiments of the present disclosure provide a query method, device and storage medium based on natural language and data knowledge, so as to at least solve the technical problem existing in the prior art that due to the possible errors and uncertainties in the relationship extraction and entity linking in the knowledge graph, the accuracy of the method of using the knowledge graph generated based on metadata to help users obtain data is low, which in turn affects the quality of the data and the credibility of the analysis results.

[0006] According to one aspect of an embodiment of the present disclosure, a query method based on natural language and data knowledge is provided, including: constructing a first database based on collected metadata and corresponding data lineage, and constructing a second database based on collected historical business data; determining multiple data lineages corresponding to the metadata in the first database, and determining multiple joint knowledge vectors based on the metadata and the corresponding data lineages; matching a first query vector with the multiple joint knowledge vectors, and determining a target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to a first query text of a terminal device, and the first query text is natural language text information; generating a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; and querying the second database using the first SQL language instruction to obtain and return the corresponding first business data to the terminal device.

[0007] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is running, a processor executes any one of the methods described above.

[0008] According to another aspect of an embodiment of the present disclosure, a query device based on natural language and data knowledge is also provided, including: a database construction module, used to construct a first database based on the collected metadata and the corresponding data lineage, and to construct a second database based on the collected historical business data; a joint knowledge vector determination module, used to determine multiple data lineages corresponding to the metadata in the first database, and determine multiple joint knowledge vectors based on the metadata and the corresponding data lineages; a first matching module, used to match the first query vector with the multiple joint knowledge vectors, and determine the target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to the first query text of the terminal device, and the first query text is natural language text information; an SQL language instruction generation module, used to generate a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; and a first business data return module, used to use the first SQL language instruction to query the second database, obtain and return the corresponding first business data to the terminal device.

[0009] According to another aspect of an embodiment of the present disclosure, a query device based on natural language and data knowledge is also provided, including: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: constructing a first database based on the collected metadata and the corresponding data lineage, and constructing a second database based on the collected historical business data; determining multiple data lineages corresponding to the metadata in the first database, and determining multiple joint knowledge vectors based on the metadata and the corresponding data lineages; matching the first query vector with the multiple joint knowledge vectors, and determining a target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to a first query text of a terminal device, and the first query text is natural language text information; generating a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; and using the first SQL language instruction to query the second database, obtain and return the corresponding first business data to the terminal device.

[0010] The present application provides a query method based on natural language and data knowledge. First, the database construction module constructs a first database based on the collected metadata and the corresponding data lineage, and constructs a second database based on the collected historical business data. Then, the database construction module determines multiple data lineages corresponding to the metadata in the first database, and determines multiple joint knowledge vectors based on the metadata and the corresponding data lineages. Further, the matching module matches the first query vector with the multiple joint knowledge vectors, and determines multiple target joint knowledge vectors corresponding to the first query vector. Thereafter, the generation module generates a first SQL language instruction based on the multiple target joint knowledge vectors and a pre-trained generation language model. Finally, the query module uses the first SQL language instruction to query the second database, obtain and return the corresponding first business data to the terminal device.

[0011] Referring to the above content, unlike the prior art of directly matching metadata with the first query vector or matching with the first query vector based on the knowledge graph, the present application first determines multiple joint knowledge vectors based on metadata and corresponding data lineages, and then matches the first query vector with the joint knowledge vector, and determines multiple target joint knowledge vectors corresponding to the first query vector. Among them, the matching degree of the target joint knowledge vector with the first query vector is greater than the matching degree of other joint knowledge vectors with the first query vector.

[0012] Therefore, since the present application has determined in advance a target joint knowledge vector that is strongly correlated with the first query vector among multiple joint knowledge vectors, the correlation between the target joint knowledge vector and the first query vector is stronger, whether for other joint knowledge vectors that are associated with the first query vector or for joint knowledge vectors that are not associated with the first query vector.

[0013] Therefore, since the present application finally generates the first SQL language instruction based on multiple target joint knowledge vectors, the first business data finally obtained based on the first SQL language instruction is more accurate, thereby being able to achieve the technical effect of greatly reducing query errors and increasing data quality and the credibility of analysis results.

[0014] This solves the technical problem in the prior art that the relationship extraction and entity linking in the knowledge graph may have errors and uncertainties, so the accuracy of the knowledge graph generated based on metadata and the method of helping users obtain data is low, which in turn affects the quality of the data and the credibility of the analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0016] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present disclosure;

[0017] Figure 2 is a schematic diagram of an intelligent questioning system based on metadata and data lineage according to Example 1 of the present application;

[0018] Figure 3 is a modular schematic diagram of the application server according to Embodiment 1 of the present application;

[0019] Figure 4 It is a flowchart of the intelligent data query method based on metadata and data lineage according to Example 1 of the present application;

[0020] Figure 5 This is an overall flow chart of returning the first service data to the terminal device according to Embodiment 1 of the present application;

[0021] Figure 6 This is an overall flow chart of returning the first score and the first service data to the terminal device according to Embodiment 1 of the present application;

[0022] Figure 7This is an overall flow chart of returning the third score and the second service data to the terminal device according to Embodiment 1 of the present application;

[0023] Figure 8 is a schematic diagram of a query device based on natural language and data knowledge according to Embodiment 2 of the present application; and

[0024] Fig. 9 It is a schematic diagram of a query device based on natural language and data knowledge according to Example 3 of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0027] Example 1

[0028] According to this embodiment, a method embodiment of query based on natural language and data knowledge is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device for implementing a query method based on natural language and data knowledge. Figure 1As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. A person skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0030] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computing device. As involved in the embodiments of the present disclosure, the data processing circuitry acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0031] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the intelligent query method based on metadata and data lineage in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the query method based on natural language and data knowledge of the above-mentioned application program is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0032] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0033] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0034] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0035] Figure 2 is a schematic diagram of a query system based on natural language and data knowledge according to an embodiment of the present application. Figure 2 As shown, the system includes: a terminal device 100, an application server 200, a large model server 300 running a generative language model, a first database 410, and a second database 420. The second database 420 stores the business data queried by the user. The first database 410 stores data knowledge related to the business data in the second database 420, and the data knowledge includes, for example, metadata or data element information related to the business data, which is used to describe the table structure, field name, data lineage, and corresponding rules of the business data.

[0036] also, Figure 3 The module diagram of the application server 200 is further shown. Figure 3 As shown, the application server 200 includes a user interface module, a database construction module, a retrieval module, an enhancement module, a matching module, a query module and a verification module.

[0037] The user interface module interacts with the terminal device 100, receives the text information of the data processing input by the user, and returns the result of the data processing to the terminal device 100. The retrieval module, the enhancement module and the matching module are used to interact with the first database 410 and the large model server 300, and generate SQL language instructions corresponding to the query text (for example, the first SQL language instruction or the second SQL language instruction) through the operation of RAG. The query module is also used to obtain the corresponding business data (for example, the first business data or the second business data) in the second database 420 using the generated SQL language instructions. The verification module is used to determine the score corresponding to the query vector (for example, the first score or the third score) based on the SQL language instructions and the sample pairs in the first database 410.

[0038] In addition, the application server 200 also includes a database construction module, wherein the database construction module is used to construct a first database 410 based on the collected metadata and the corresponding data lineage; the database construction module is used to construct a second database 420 based on the collected historical business data. Among them, metadata includes data collection, data model, data assets, indicator data, label portrait data, data report and data works, and data metadata includes version, name, description and address information. Data lineage includes information such as data link relationship and data value.

[0039] It should be noted that the terminal device 100, application server 200, large model server 300, first database 410 and second database 420 in the system can all be applied to the hardware structure described above.

[0040] In the above operating environment, according to the first aspect of this embodiment, a query method based on natural language and data knowledge is provided. The method comprises Figure 2 The application server 200 shown in FIG. Figure 4 A schematic diagram showing the process of the method is shown in FIG. Figure 4 As shown, the method includes:

[0041] S402: constructing a first database based on the collected metadata and corresponding data lineage, and constructing a second database based on the collected historical business data;

[0042] S404: Determine a plurality of data lineages corresponding to the metadata in the first database, and determine a plurality of joint knowledge vectors based on the metadata and the corresponding data lineages;

[0043] S406: Matching the first query vector with a plurality of joint knowledge vectors, and determining a target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to a first query text of the terminal device, and the first query text is natural language text information;

[0044] S408: Generate a first SQL language instruction based on the target joint knowledge vector and the pre-trained generative language model; and

[0045] S410: Use the first SQL language instruction to query in the second database, obtain and return the corresponding first business data to the terminal device.

[0046] Specifically, Figure 5 This is an overall flow chart of returning the first service data to the terminal device according to an embodiment of the present application. Figure 5 As shown, first, before the application server 200 receives the query text information sent by the terminal device 100 through the user interface module, the data construction module pre-collects metadata and data lineage corresponding to the metadata. The metadata may include, for example, table structure data and field name data. Thus, the data lineage corresponding to the metadata includes the table structure data lineage corresponding to the table structure data and the field name data lineage corresponding to the field name data. Afterwards, the data construction module collects historical business data, wherein the historical business data may be, for example, business data that the user has queried.

[0047] Then, the data construction module constructs a first database based on the collected metadata and the corresponding data lineage, and constructs a second database based on the collected historical business data (S402). The first database includes, for example, three data modules A, B, and C. And the A data module can, for example, consist of multiple table structure data A Q,1 ~A Q,n And with each table structure data A Q,1 ~A Q,n The corresponding table structure vector A X,1 ~A X,n The B data module can be composed of multiple field name data B Q,1 ~B Q,m And the data B with each field name Q,1 ~B Q,m The corresponding field name vector B X,1 ~B X,m composition.

[0048] And the C data module may be composed of multiple data lineages and data lineage vectors corresponding to each data lineage. The multiple data lineages include the data A corresponding to the table structure. Q,1 ~A Q,n Corresponding data lineage And the data B with each field name Q,1 ~B Q,m Corresponding Thus, the data lineage vectors corresponding to each data lineage include the table structure vector A X,1 ~A X,n The corresponding data lineage vector and the vector B with each field name X,1 ~B X,m The corresponding data lineage vector

[0049] Further, the data construction module determines a plurality of data lineages corresponding to the metadata in the first database, and determines a plurality of joint knowledge vectors based on the metadata and the corresponding data lineages (S404). Referring to the above, it can be seen that the metadata in the first database includes the table structure data in the A data module, and the data lineage in the C data module includes the data lineage corresponding to the table structure data, so that the database construction module can determine the table structure data and the data lineage corresponding to the respective table structure data.

[0050] For example, with table structure data A Q,1 The corresponding data lineage is With table structure data A Q,2 The corresponding data lineage is And so on; with table structure data A Q,n The corresponding data lineage is

[0051]

[0052] Thus, the database construction module can generate the joint knowledge text corresponding to the table structure data when determining the table structure data and the data lineage corresponding to each table structure data. For example, the joint knowledge text corresponding to the table structure data Including table structure data A Q,1 And the corresponding data lineage Table structure data A Q,1 And the corresponding data lineage Table structure data A Q,1 And the corresponding data lineage And table structure data A Q,1 And the corresponding data lineage Other joint knowledge texts corresponding to table structure data The data composition form is the same as the above joint knowledge text The data composition format is the same, so it will not be repeated here.

[0053] And similar to the above, since the metadata in the first database includes the field name data in the B data module, and the data lineage in the C data module includes the data lineage corresponding to the field name data, the database construction module can determine the field name data and the data lineage corresponding to each field name data.

[0054] For example, with field name dataB Q,1 The corresponding data lineage is With field name data B Q,2 The corresponding data lineage is And so on; with field name data B Q,m The corresponding data lineage is

[0055]

[0056] Thus, the database construction module can generate the joint knowledge text corresponding to the field name data when determining the field name data and the data lineage corresponding to each field name data. For example, the joint knowledge text corresponding to the field name data Include field name data B Q,1 And the corresponding data lineage Field Name Data B Q,1 And the corresponding data lineage And field name data B Q,1 And the corresponding data lineage Other joint knowledge text corresponding to field name data The data composition form is the same as the above joint knowledge text The data composition format is the same, so it will not be repeated here.

[0057] That is, when the database construction module generates the joint knowledge text corresponding to the table structure data and the joint knowledge text corresponding to the field name data, a plurality of joint knowledge texts D corresponding to the metadata can be determined. Q,1 ~D Q,u . Where u = k + b. And multiple joint knowledge texts D are determined in the database construction module Q,1 ~D Q,u In the case of Q,1 ~D Q,u Perform embedding operations to determine multiple joint knowledge vectors D X,1 ~D X,u .

[0058] Furthermore, the application server 200 may receive, for example, through a user interface module, query text information uploaded by the user through the terminal device 100. The query text information may be, for example, a txt file or a json file, and the query text information may be, for example, natural language text information related to data query.

[0059] Afterwards, the matching module vectorizes the query text information corresponding to the terminal device 100 using the pre-trained embedding model and generates a first query vector. After the matching module determines the first query vector, the first query vector is matched with the multiple joint knowledge vectors in the first database 410, and the target joint knowledge vector corresponding to the first query vector is determined (S406). Specifically, as can be seen from the above content, the database construction module has already pre-determined multiple joint knowledge vectors D corresponding to the metadata. X,1 ~D X,u In the case of, the matching module includes but is not limited to calculating the similarity between the first query vector and the multiple joint knowledge vectors using vector similarity calculation methods such as cosine similarity, Pearson correlation coefficient and Euclidean distance. And when the similarity between the first query vector and any one or more of the multiple joint knowledge vectors is greater than a first preset threshold, the corresponding joint knowledge vector is determined as the target joint knowledge vector. Otherwise, the joint knowledge vectors whose similarity is less than the first preset threshold are eliminated.

[0060] Therefore, it can be seen from the above content that the correlation between the one or more target joint knowledge vectors and the first query vector finally determined by the matching module is stronger than the correlation between other joint knowledge vectors and the first query vector.

[0061] The generation module then generates a first SQL language instruction (S408) based on the target joint knowledge vector and the pre-trained generative language model. Specifically, first, the generation module determines the context information (context) corresponding to the first query text, wherein the context information is used to indicate the target joint knowledge text corresponding to one or more target joint knowledge vectors. Then, the enhancement module combines the first query text with the corresponding context information and generates corresponding enhancement information. Then, the enhancement module fills the enhancement information into a pre-set prompt template and generates a second query text. Finally, the generation module inputs the second query text into the pre-trained generative language model to generate a first SQL language instruction. The above content will be described in detail later, so it will not be repeated here.

[0062] Finally, the query module inputs the first SQL language instruction into the second database 420 storing the business data for query, thereby obtaining the first business data corresponding to the first SQL language instruction. When the query module obtains the first business data, the application server 200 returns the first business data to the terminal device 100 by using the user interface module (S410).

[0063] As described in the background technology, there are specific examples of using metadata to generate knowledge graphs in the prior art. However, there are still many defects in using knowledge graphs for data query. For example, since there may be errors and uncertainties in the relationship extraction and entity linking in the knowledge graph, the accuracy of using the knowledge graph generated based on metadata to help users obtain data is low, which in turn affects the quality of the data and the credibility of the analysis results.

[0064] In view of this, the present application provides a query method based on natural language and data knowledge. And unlike the prior art of directly matching metadata with the first query vector or matching with the first query vector based on the knowledge graph, the present application first determines multiple joint knowledge vectors based on metadata and the corresponding data lineage, and then matches the first query vector with the joint knowledge vector, and determines multiple target joint knowledge vectors corresponding to the first query vector. Among them, the matching degree of the target joint knowledge vector with the first query vector is greater than the matching degree of other joint knowledge vectors with the first query vector.

[0065] Therefore, since the present application has determined in advance a target joint knowledge vector that is strongly correlated with the first query vector among multiple joint knowledge vectors, the correlation between the target joint knowledge vector and the first query vector is stronger, whether for other joint knowledge vectors that are associated with the first query vector or for joint knowledge vectors that are not associated with the first query vector.

[0066] Therefore, since the present application finally generates the first SQL language instruction based on multiple target joint knowledge vectors, the first business data finally obtained based on the first SQL language instruction is more accurate, thereby being able to achieve the technical effect of greatly reducing query errors and increasing data quality and the credibility of analysis results.

[0067] This solves the technical problem in the prior art that the relationship extraction and entity linking in the knowledge graph may have errors and uncertainties, so the accuracy of the knowledge graph generated based on metadata and the method of helping users obtain data is low, which in turn affects the quality of the data and the credibility of the analysis results.

[0068] Optionally, the operation of generating a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model includes: determining context information corresponding to the first query text, wherein the context information is used to indicate the target joint knowledge text corresponding to multiple target joint knowledge vectors; combining the first query text with the corresponding context information and generating corresponding enhanced information; filling the enhanced information into a pre-set prompt template and generating a corresponding second query text; and inputting the second query text into the generative language model and generating a first SQL language instruction.

[0069] Specifically, when the matching module of the application server 200 determines one or more target joint knowledge vectors corresponding to the first query vector, the enhancement module of the application server 200 determines context information corresponding to the first query text. The context information is used to indicate the target joint knowledge text corresponding to the multiple target joint knowledge vectors, so that the context information includes metadata and data lineage corresponding to the metadata.

[0070] Then the enhancement module combines the first query text with the corresponding context information to obtain corresponding enhancement information. Further, the generation module fills the enhancement information corresponding to the first query text into a preset prompt template and generates a second query text.

[0071] Finally, the generation module inputs the second query text into the pre-trained generative language model and generates a first SQL language instruction.

[0072] Thus, by taking the target joint knowledge text as the context information corresponding to the first query text, and combining the first query text with the context information to generate the second query text, and based on the second query text, generating the operation of the first SQL language instruction, a more accurate first SQL language instruction can be generated, thereby ensuring a more accurate technical effect of the first business data subsequently returned to the terminal device 100.

[0073] Optionally, the operation of matching the first query vector with multiple joint knowledge vectors and determining a target joint knowledge vector corresponding to the first query vector includes: matching the first query vector with multiple joint knowledge vectors respectively and determining corresponding matching results; determining the target joint knowledge vector corresponding to the first query vector based on the matching results and a first preset threshold; determining the joint knowledge vector as the target joint knowledge vector when the matching result is greater than or equal to the first preset threshold; and eliminating the joint knowledge vector when the matching result is less than the first preset threshold.

[0074] Specifically, after the application server 200 receives the first query text corresponding to the terminal device 100 through the user interface module and vectorizes the first query text using the embedding model, the application server 200 uses the matching module to match the first query vector with multiple joint knowledge vectors respectively.

[0075] For example, referring to the above contents, it can be known that the first database 410 includes multiple joint knowledge texts D Q,1 ~D Q,u Corresponding to multiple joint knowledge vectors D X,1 ~D X,u Thus, the matching module inputs the first query vector into the first database 410, and uses the vector similarity algorithm to compare the first query vector with each joint knowledge vector D X,1 ~D X,u Thus, the matching module can determine the first query vector and each joint knowledge vector D X,1 ~D X,u The similarity value between S1 and S u . The matching module will then be combined with each joint knowledge vector D X,1 ~D X,u The corresponding similarity values ​​S1~S u (i.e., the matching result) and the first preset threshold S e Compare and convert the similarity values ​​S1 to S u Greater than or equal to the first preset threshold S e The joint knowledge vector of is determined as the target joint knowledge vector; the similarity values ​​S1~S u Less than the first preset threshold S e The joint knowledge vector of is eliminated.

[0076] For example, after matching the first query vector with each joint knowledge vector, the matching module determines the first query vector and the joint knowledge vector D X,1 ~D X,u The corresponding similarity values ​​S1~S u And the matching module can determine after the matching operation, and the joint knowledge vector D X,1 ~D X,3 The corresponding similarity values ​​S1-S3 are greater than the first preset threshold S e Thus, the joint knowledge vector D X,1 ~D X,3 That is, the target joint knowledge vector corresponding to the first query vector.

[0077] Thus, by matching the first query vector with each joint knowledge vector respectively, and determining the joint knowledge vector having a matching degree greater than a first preset threshold as the target joint knowledge vector, the technical effect of being able to screen out the target joint knowledge vector having a strong correlation with the first query vector is achieved.

[0078] Optionally, multiple data lineages corresponding to the metadata in the first database are determined, and based on the metadata and the corresponding data lineages, the operations of determining multiple joint knowledge vectors include: determining multiple data lineages corresponding to the metadata, and combining the metadata with the corresponding data lineages to form multiple joint knowledge texts; and using a pre-trained embedding model to perform embedding operations on each joint knowledge text, thereby determining multiple joint knowledge vectors respectively.

[0079] Specifically, before the matching module uses the multiple joint knowledge vectors in the first database 410 to match the first query vector, the database construction module needs to construct multiple joint knowledge vectors in the first database 410 in advance.

[0080] Thus, first, the database construction module determines multiple data lineages corresponding to the metadata. Referring to the above content, it can be seen that the metadata in the first database 410 includes table structure data A Q,1 ~A Q,n And field name data B Q,1 ~B Q,m The data lineage in the first database 410 includes the table structure data A Q,1 ~A Q,n Corresponding data lineage and data with field name B Q,1 ~B Q,m Corresponding data lineage

[0081]

[0082] Then, the database construction module combines the metadata with the corresponding data lineage to form multiple joint knowledge texts. Specifically, first, the database construction module determines the data lineage corresponding to each table structure data. For example, Q,1 The corresponding data lineage is With table structure data A Q,2 The corresponding data lineage is And so on; with table structure data A Q,n The corresponding data lineage is Then, the database construction module determines the joint knowledge text based on the correspondence between the table structure data and the data lineage. For example, the joint knowledge text corresponding to the table structure data Including table structure data A Q,1And the corresponding data lineage Table structure data A Q,1 And the corresponding data lineage Table structure data A Q,1 And the corresponding data lineage And table structure data A Q,1 And the corresponding data lineage Other joint knowledge texts corresponding to table structure data The data composition form is the same as the above joint knowledge text The data composition format is the same, so it will not be repeated here.

[0083] Finally, the database construction module uses the pre-trained embedding model to embed each joint knowledge text D Q,1 ~D Q,u Perform embedding operations to determine multiple joint knowledge vectors D X,1 ~D X,u . Wherein, u=k+b.

[0084] Thus, by utilizing the metadata and data lineage collected in advance, and based on the metadata and data lineage, generating multiple joint knowledge texts and joint knowledge vectors corresponding to each joint knowledge text, the technical effect of helping to establish a multi-dimensional three-dimensional relationship of data and making the subsequently generated first SQL language instructions more accurate is achieved.

[0085] Optionally, the operation of constructing the first database based on the collected metadata and the corresponding data lineage includes: collecting table structure data, and determining a table structure vector corresponding to the table structure data based on the table structure data; collecting field name data, and determining a field vector corresponding to the field name data based on the field name data; determining the corresponding table structure data lineage based on the table structure data, and determining the corresponding field name data lineage based on the field name data; and constructing the first database based on the table structure data, the table structure vector, the table structure data lineage corresponding to the table structure data, the field name data, the field vector and the field data lineage corresponding to the field name data.

[0086] Specifically, before the application server 200 receives the first query text through the user interface module, the application server 200 constructs the first database 410 in advance through the database construction module. Q,1 ~A Q,n The table structure data A Q,1 ~A Q,nThis includes but is not limited to the table structure information crawled from the web page by the database construction module, or the historical table structure information sent by the user to the application server 200 through the terminal device 100. Q,1 ~A Q,n , and use the pre-trained embedding model to determine the table structure data A Q,1 ~A Q,n The corresponding table structure vector A X,1 ~A X,n .

[0087] For example, table structure data A Q,1 Corresponding to the table structure vector A X,1 ; Table structure data A Q,2 Corresponding to the table structure vector A X,2 ; And so on; Table structure data A Q,n Corresponding to the table structure vector A X,n . Thus, the database construction module can be based on the table structure data A Q,1 ~A Q,n And the corresponding table structure vector A X,1 ~A X,n , data module A is generated in the first database 410.

[0088] The same as above is that the database construction module collects field name data B Q,1 ~B Q,m . The field name data B Q,1 ~B Q,m This includes but is not limited to the field name information crawled from the web page by the database construction module, or the historical field name information sent by the user to the application server 200 through the terminal device 100. Q,1 ~B Q,m , and use the pre-trained embedding model to determine the field name data B Q,1 ~B Q,m The corresponding field name vector B X,1 ~B X,m .

[0089] For example, field name dataB Q,1 Corresponding to the field name vector B X,1 ; Field name data B Q,2 Corresponding to the field name vector B X,2 ; And so on; Field name data B Q,n Corresponding to the field name vector B X,m . Thus, the database construction module can be based on the field name data B Q,1 ~B Q,m And the corresponding field name vector B X,1 ~BX,m , a data module B is generated in the first database 410 .

[0090] Furthermore, the database construction module is based on the table structure data A Q,1 ~A Q,n , determine the corresponding table structure data lineage Each table structure data corresponds to one or more data lineages. And the database construction module is based on the field name data B Q,1 ~B Q,m , determine the corresponding field name data lineage Each field name data corresponds to one or more data lineages. Therefore, the data construction module can be based on the table structure data lineage And field name data lineage A data module C is generated in the first database 410 .

[0091] Finally, the database construction module can construct the first database 410 based on the above data module A, data module B and data module C.

[0092] Optionally, it also includes: using an embedding model to vectorize the first SQL language instruction and determine the corresponding first SQL language instruction vector; matching the first SQL language instruction vector with multiple sample pairs in the first database and determining the corresponding first target sample pair, wherein the sample pair includes multiple pre-collected third query texts and second SQL language instructions corresponding to the third query texts; extracting the corresponding fourth query text from the first target sample pair, and using the embedding model to vectorize the fourth query text to generate a second query vector; determining the similarity between the second query vector and the first query vector, and generating a first score corresponding to the second query vector; and when the first score is greater than or equal to a second preset threshold, returning the first score and the first business data to the terminal device.

[0093] From the above content, it can be known that in addition to the above application server 200 directly returning the first business data corresponding to the first SQL language instruction to the terminal device 100 through the user interface module, the present application also provides a quality verification method for the first SQL language instruction. Specifically, before the query module uses the first SQL language instruction to query in the second database 420, obtain and return the corresponding first business data to the terminal device 100, the application server 200 can also use the verification module to verify the quality of the first SQL language instruction.

[0094] Figure 6 This is an overall flow chart of returning the first score and the first service data to the terminal device according to an embodiment of the present application. Figure 6As shown, first, the application server 200 vectorizes the first SQL language instruction using a pre-trained embedding model and determines a corresponding first SQL language instruction vector.

[0095] Then, the matching module inputs the first SQL language instruction vector into the first database 410, and matches the first SQL language instruction vector with a plurality of sample pairs in the first database 410. When a matching result is generated, the matching result is compared with a preset threshold, and a sample pair whose matching result is greater than or equal to the preset threshold is determined as a first target sample pair. Each sample pair includes a pre-collected third query text and a corresponding second SQL language instruction.

[0096] In addition, it is worth noting that before matching the first SQL language instruction vector with multiple sample pairs, the database construction module needs to collect multiple third query texts and second SQL language instructions corresponding to each third query text in advance. The multiple third query texts can be obtained by the database construction module crawling from a web page, or collecting historical query information of multiple users. Thus, multiple sample pairs corresponding to the third query text-second SQL language instruction can be generated in the first database 410, and the sample pairs corresponding to the multiple third query texts-second SQL language instructions in the first database 410 are included as data module E.

[0097] Further, after the matching module determines the first target sample pair corresponding to the first SQL language instruction, the application server 200 uses the generation module to extract the corresponding fourth query text from the first target sample pair, and uses the embedding model to vectorize the fourth query text, thereby generating a second query vector. For example, the sample pair E1 is determined as the first target sample pair, so that the generation module extracts the fourth query text E from the sample pair E1. f1 Then the generation module uses the embedding model to embed multiple fourth query texts E f1 Vectorization is performed to generate a second query vector.

[0098] The verification module then determines the similarity between the second query vector and the first query vector, and generates a first score corresponding to the second query vector. For example, the verification module calculates the similarity between the second query vector and the first query vector using a vector similarity algorithm, and uses the calculation result as the first score corresponding to the second query vector.

[0099] Finally, when the query module has used the first SQL language instruction to query and obtain the first business data in the second database 420 , the application server 200 returns the first business data and the first score to the terminal device 100 .

[0100] Thus, when the user receives the first business data and the first score through the terminal device 100, the quality of the first SQL language instruction generated by the application server 200 can be determined based on the first score, and the quality of the generated first business data can be further determined, and the technical effect of the analysis standard that ensures that the user can have visibility of the generated first business data can be achieved.

[0101] Optionally, it also includes: when the score corresponding to the first query vector is less than a second preset threshold, matching the first query vector with the metadata in the first database, and generating a third SQL language instruction based on the matching result; and vectorizing the third SQL language instruction using an embedding model, and determining a corresponding second SQL language instruction vector; matching the second SQL language instruction vector with multiple sample pairs in the first database, and determining a corresponding second target sample pair; extracting a corresponding fifth query text from the second target sample pair, and vectorizing the fifth query text using an embedding model to generate a third query vector; determining the similarity between the third query vector and the first query vector, and generating a second score corresponding to the third query vector; and determining a third score, and returning the third score and second business data corresponding to the third score to the terminal device, wherein the third score is used to indicate a larger score between the first score and the second score.

[0102] Specifically, Figure 7 This is an overall flow chart of returning the third score and the second service data to the terminal device according to an embodiment of the present application. Figure 7 As shown, when the first score is less than the second preset threshold, it means that the accuracy of the first business data corresponding to the first score is low, so the matching module inputs the first query vector into the first database 410, and uses the A data module and the B data module in the first database 410 (that is, the metadata in the first database 410) to match the first query vector, thereby determining the target table structure vector and the target field name vector corresponding to the first query vector. Then the generation module generates a third SQL language instruction based on the target table structure vector and the target field name vector. It is worth noting that the above method of generating the third SQL language instruction is the same as the method of generating the first SQL language instruction, so it will not be repeated here.

[0103] Then, the generation module vectorizes the third SQL language instruction using the pre-trained embedding model and determines the corresponding second SQL language instruction vector. Further, the matching module matches the second SQL language instruction vector with a plurality of sample pairs in the first database 410 and determines the corresponding second target sample pair. It is worth noting that the above-mentioned method of determining the second target sample pair is the same as the method of determining the first target sample pair, so it will not be repeated here.

[0104] Further, the generation module extracts the corresponding fifth query text from the second target sample pair, and vectorizes the fifth query text using the pre-trained embedding model to generate a third query vector. The generation module then determines the similarity between the third query vector and the first query vector, and generates a second score corresponding to the third query vector.

[0105] Thus, the first score is used to indicate the similarity between the second query vector and the first query vector, and the second score is used to indicate the similarity between the third query vector and the first query vector. The similarity between the first query vector and the second query vector corresponds to the target joint knowledge text, and the similarity between the third query vector and the first query vector corresponds to the target table structure data and the target field name data.

[0106] And when the generation module determines the first score and the second score, the larger score of the first score and the second score is further determined. The larger the score, the higher the similarity between the generated SQL language instruction and the first query text (i.e., the query text input by the user to the application server 200 through the terminal device 100), and the more accurate the business data finally generated is.

[0107] Finally, the application server 200 returns the third score and the second business data corresponding to the third score to the terminal device 100 through the user interface module. For example, when the first score is large, it means that the similarity between the first SQL language instruction corresponding to the first score and the first query text is high, and the first business data corresponding to the first SQL language instruction is relatively accurate; when the second score is large, it means that the similarity between the third SQL language instruction corresponding to the second score and the first query text is high, and the third business data corresponding to the third SQL language instruction is relatively accurate. Among them, the third business data is generated by the generation module based on the metadata, and the generation method is the same as the target joint knowledge text, which will not be repeated here.

[0108] Thus, by comparing the first score (corresponding to the target joint knowledge text) with the second score (corresponding to the target table structure data and the target field name data), and selecting the business data corresponding to the larger score as the operation returned to the terminal device 100, the technical effect of ensuring the accuracy of the business data returned to the terminal device 100 is achieved.

[0109] Therefore, according to the first aspect of this embodiment, since the present application finally generates a first SQL language instruction based on multiple target joint knowledge vectors, the first business data finally obtained based on the first SQL language instruction is more accurate, thereby being able to achieve the technical effect of greatly reducing query errors and increasing data quality and the credibility of analysis results.

[0110] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0111] Therefore, according to this embodiment, since the present application finally generates a first SQL language instruction based on multiple target joint knowledge vectors, the first business data finally obtained based on the first SQL language instruction is more accurate, thereby being able to achieve the technical effect of greatly reducing query errors and increasing data quality and the credibility of analysis results.

[0112] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0114] Example 2

[0115] Figure 8 FIG. 8 shows a query device 800 based on natural language and data knowledge according to this embodiment, which corresponds to the method according to embodiment 1. Figure 8As shown, the device 800 includes: a database construction module 810, which is used to construct a first database based on the collected metadata and the corresponding data lineage, and to construct a second database based on the collected historical business data; a joint knowledge vector determination module 820, which is used to determine multiple data lineages corresponding to the metadata in the first database, and determine multiple joint knowledge vectors based on the metadata and the corresponding data lineages; a first matching module 830, which is used to match the first query vector with the multiple joint knowledge vectors, and determine the target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to the first query text of the terminal device, and the first query text is natural language text information; an SQL language instruction generation module 840, which is used to generate a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; and a first business data return module 850, which is used to query the second database using the first SQL language instruction, and obtain and return the corresponding first business data to the terminal device.

[0116] Optionally, the SQL language instruction generation module 840 includes: a context information determination module, used to determine the context information corresponding to the first query text, wherein the context information is used to indicate the target joint knowledge text corresponding to multiple target joint knowledge vectors; an enhancement information generation module, used to combine the first query text with the corresponding context information and generate corresponding enhancement information; a second query text generation module, used to fill the enhancement information into a pre-set prompt template and generate a corresponding second query text; and an SQL language instruction generation sub-module, used to input the second query text into the generative language model and generate a first SQL language instruction.

[0117] Optionally, the first matching module 830 includes: a matching result determination module, used to match the first query vector with multiple joint knowledge vectors respectively, and determine the corresponding matching results; a target joint knowledge vector determination module, used to determine the target joint knowledge vector corresponding to the first query vector based on the matching result and a first preset threshold; a first matching result generation module, used to determine the joint knowledge vector as the target joint knowledge vector when the matching result is greater than or equal to the first preset threshold; and a second matching result generation module, used to eliminate the joint knowledge vector when the matching result is less than the first preset threshold.

[0118] Optionally, the joint knowledge vector determination module 820 includes: a joint knowledge text generation module, which is used to determine multiple data lineages corresponding to the metadata, and respectively combine the metadata with the corresponding data lineages to form multiple joint knowledge texts; and a joint knowledge vector determination sub-module, which is used to use a pre-trained embedding model to perform embedding operations on each joint knowledge text, thereby respectively determining multiple joint knowledge vectors.

[0119] Optionally, the database construction module 810 includes: a table structure data collection module, which is used to collect table structure data and determine the table structure vector corresponding to the table structure data based on the table structure data; a field name data collection module, which is used to collect field name data and determine the field name vector corresponding to the field name data based on the field name data; a data lineage determination module, which is used to determine the corresponding table structure data lineage based on the table structure data and determine the corresponding field name data lineage based on the field name data; and a database construction sub-module, which is used to construct a first database based on the table structure data, the table structure vector, the table structure data lineage corresponding to the table structure data, the field name data, the field name vector and the field name data lineage corresponding to the field name data.

[0120] Optionally, the device 800 also includes: a first SQL language instruction vector determination module, which is used to vectorize the first SQL language instruction using an embedded model and determine the corresponding first SQL language instruction vector; a first target sample pair determination module, which is used to match the first SQL language instruction vector with multiple sample pairs in the first database and determine the corresponding first target sample pair, wherein the sample pair includes multiple pre-collected third query texts and second SQL language instructions corresponding to the third query texts; a second query vector generation module, which is used to extract the corresponding fourth query text from the first target sample pair and vectorize the fourth query text using the embedded model to generate a second query vector; a first score generation module, which is used to determine the similarity between the second query vector and the first query vector and generate a first score corresponding to the second query vector; and a score return module, which is used to return the first score and the first business data to the terminal device when the first score is greater than or equal to the second preset threshold.

[0121] Optionally, the device 800 also includes: a third SQL language instruction generation module, which is used to match the first query vector with the metadata in the first database when the first score is less than the second preset threshold, and generate a third SQL language instruction based on the matching result; a second SQL language instruction vector determination module, which is used to vectorize the third SQL language instruction using the embedding model and determine the corresponding second SQL language instruction vector; a second target sample pair determination module, which is used to match the second SQL language instruction vector with multiple sample pairs in the first database and determine the corresponding second target sample pair; a third query vector generation module, which is used to extract the corresponding fifth query text from the second target sample pair, and vectorize the fifth query text using the embedding model to generate a third query vector; a second score determination module, which is used to determine the similarity between the third query vector and the first query vector, and generate a second score corresponding to the third query vector; and a second business data return module, which is used to determine the third score, and return the third score and the second business data corresponding to the third score to the terminal device, wherein the third score is used to indicate the larger score between the first score and the second score.

[0122] Therefore, according to this embodiment, since the present application finally generates a first SQL language instruction based on multiple target joint knowledge vectors, the first business data finally obtained based on the first SQL language instruction is more accurate, thereby being able to achieve the technical effect of greatly reducing query errors and increasing data quality and the credibility of analysis results.

[0123] Example 3

[0124] Fig. 9 FIG. 9 shows a query device 900 based on natural language and data knowledge according to this embodiment, which corresponds to the method according to embodiment 1. Fig. 9 As shown, the device 900 includes: a processor 910; and a memory 920, which is connected to the processor 910 and is used to provide instructions for the processor 910 to process the following processing steps: construct a first database based on the collected metadata and the corresponding data lineage, and construct a second database based on the collected historical business data; determine multiple data lineages corresponding to the metadata in the first database, and determine multiple joint knowledge vectors based on the metadata and the corresponding data lineages; match the first query vector with the multiple joint knowledge vectors, and determine a target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to the first query text of the terminal device, and the first query text is natural language text information; generate a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; and use the first SQL language instruction to query the second database, obtain and return the corresponding first business data to the terminal device.

[0125] Optionally, the operation of generating a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model includes: determining context information corresponding to the first query text, wherein the context information is used to indicate the target joint knowledge text corresponding to multiple target joint knowledge vectors; combining the first query text with the corresponding context information and generating corresponding enhanced information; filling the enhanced information into a pre-set prompt template and generating a corresponding second query text; and inputting the second query text into the generative language model and generating a first SQL language instruction.

[0126] Optionally, the operation of matching the first query vector with multiple joint knowledge vectors and determining a target joint knowledge vector corresponding to the first query vector includes: matching the first query vector with multiple joint knowledge vectors respectively and determining corresponding matching results; determining the target joint knowledge vector corresponding to the first query vector based on the matching results and a first preset threshold; determining the joint knowledge vector as the target joint knowledge vector when the matching result is greater than or equal to the first preset threshold; and eliminating the joint knowledge vector when the matching result is less than the first preset threshold.

[0127] Optionally, multiple data lineages corresponding to the metadata in the first database are determined, and based on the metadata and the corresponding data lineages, the operations of determining multiple joint knowledge vectors include: determining multiple data lineages corresponding to the metadata, and combining the metadata with the corresponding data lineages to form multiple joint knowledge texts; and using a pre-trained embedding model to perform embedding operations on each joint knowledge text, thereby determining multiple joint knowledge vectors respectively.

[0128] Optionally, the operation of constructing the first database based on the collected metadata and the corresponding data lineage includes: collecting table structure data, and determining the table structure vector corresponding to the table structure data based on the table structure data; collecting field name data, and determining the field name vector corresponding to the field name data based on the field name data; determining the corresponding table structure data lineage based on the table structure data, and determining the corresponding field name data lineage based on the field name data; and constructing the first database based on the table structure data, the table structure vector, the table structure data lineage corresponding to the table structure data, the field name data, the field name vector and the field name data lineage corresponding to the field name data.

[0129] Optionally, the device 900 also includes: using an embedding model to vectorize the first SQL language instruction and determine the corresponding first SQL language instruction vector; matching the first SQL language instruction vector with multiple sample pairs in the first database and determining the corresponding first target sample pair, wherein the sample pair includes multiple pre-collected third query texts and second SQL language instructions corresponding to the third query texts; extracting the corresponding fourth query text from the first target sample pair, and vectorizing the fourth query text using the embedding model to generate a second query vector; determining the similarity between the second query vector and the first query vector, and generating a first score corresponding to the second query vector; and when the first score is greater than or equal to a second preset threshold, returning the first score and the first business data to the terminal device.

[0130] Optionally, the device 900 also includes: when the first score is less than a second preset threshold, matching the first query vector with the metadata in the first database, and generating a third SQL language instruction based on the matching result; vectorizing the third SQL language instruction using an embedding model, and determining a corresponding second SQL language instruction vector; matching the second SQL language instruction vector with multiple sample pairs in the first database, and determining a corresponding second target sample pair; extracting a corresponding fifth query text from the second target sample pair, and vectorizing the fifth query text using an embedding model to generate a third query vector; determining the similarity between the third query vector and the first query vector, and generating a second score corresponding to the third query vector; and determining a third score, and returning the third score and second business data corresponding to the third score to the terminal device, wherein the third score is used to indicate a larger score between the first score and the second score.

[0131] Therefore, according to this embodiment, since the present application finally generates a first SQL language instruction based on multiple target joint knowledge vectors, the first business data finally obtained based on the first SQL language instruction is more accurate, thereby being able to achieve the technical effect of greatly reducing query errors and increasing data quality and the credibility of analysis results.

[0132] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0133] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0138] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A query method based on natural language and data knowledge, characterized in that: include: Based on the collected metadata and the corresponding data lineage, a first database is constructed, and based on the collected historical business data, a second database is constructed; Determining a plurality of data lineages corresponding to the metadata in the first database, and determining a plurality of joint knowledge vectors based on the metadata and the corresponding respective data lineages; Matching a first query vector with the multiple joint knowledge vectors, and determining a target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to a first query text of a terminal device, and the first query text is natural language text information; Generate a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; as well as The first SQL language instruction is used to query the second database, obtain and return the corresponding first business data to the terminal device, wherein The operation of generating a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model includes: Determining context information corresponding to the first query text, wherein the context information is used to indicate a target joint knowledge text corresponding to the target joint knowledge vector; combining the first query text with corresponding context information and generating corresponding enhanced information; Filling the enhanced information into a preset prompt template and generating a corresponding second query text; and The second query text is input into the generative language model, and the first SQL language instruction is generated.

2. The method according to claim 1, characterized in that: The operation of matching the first query vector with the multiple joint knowledge vectors and determining a target joint knowledge vector corresponding to the first query vector includes: Matching the first query vector with the multiple joint knowledge vectors respectively, and determining corresponding matching results; Based on the matching result and a first preset threshold, determining a target joint knowledge vector corresponding to the first query vector; In a case where the matching result is greater than or equal to the first preset threshold, determining the joint knowledge vector as the target joint knowledge vector; and When the matching result is smaller than a first preset threshold, the joint knowledge vector is eliminated.

3. The method according to claim 1, characterized in that The operation of determining a plurality of data lineages corresponding to the metadata in the first database and determining a plurality of joint knowledge vectors based on the metadata and the corresponding data lineages includes: Determine a plurality of data lineages corresponding to the metadata, and form a plurality of joint knowledge texts by combining the metadata with the corresponding data lineages; and By using a pre-trained embedding model, an embedding operation is performed on each joint knowledge text, thereby respectively determining the multiple joint knowledge vectors.

4. The method according to claim 1, characterized in that: The operation of constructing the first database based on the collected metadata and the corresponding data lineage includes: Collecting table structure data, and determining a table structure vector corresponding to the table structure data based on the table structure data; Collecting field name data, and determining a field name vector corresponding to the field name data based on the field name data; Based on the table structure data, determining the corresponding table structure data lineage, and based on the field name data, determining the corresponding field name data lineage; and The first database is constructed based on the table structure data, the table structure vector, the table structure data lineage corresponding to the table structure data, the field name data, the field name vector and the field name data lineage corresponding to the field name data.

5. The method according to claim 1, characterized in that: Also includes: vectorizing the first SQL language instruction by using the embedded model, and determining a corresponding first SQL language instruction vector; Matching the first SQL language instruction vector with a plurality of sample pairs in the first database, and determining a corresponding first target sample pair, wherein the sample pair includes a plurality of pre-collected third query texts and a second SQL language instruction corresponding to the third query texts; Extracting a corresponding fourth query text from the first target sample pair, and vectorizing the fourth query text using the embedding model to generate a second query vector; determining a similarity between the second query vector and the first query vector, and generating a first score corresponding to the second query vector; as well as When the first score is greater than or equal to a second preset threshold, the first score and the first service data are returned to the terminal device.

6. The method according to claim 5, characterized in that Also includes: When the first score is less than the second preset threshold, matching the first query vector with metadata in the first database, and generating a third SQL language instruction based on the matching result; Vectorizing the third SQL language instruction by using the embedded model, and determining a corresponding second SQL language instruction vector; Matching the second SQL language instruction vector with a plurality of sample pairs in the first database, and determining corresponding second target sample pairs; Extracting a corresponding fifth query text from the second target sample pair, and vectorizing the fifth query text using the embedding model to generate a third query vector; determining a similarity between the third query vector and the first query vector, and generating a second score corresponding to the third query vector; as well as A third score is determined, and the third score and second service data corresponding to the third score are returned to the terminal device, wherein the third score is used to indicate a larger score between the first score and the second score.

7. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 6.

8. A query device based on natural language and data knowledge, characterized in that: include: A database construction module, used to construct a first database based on the collected metadata and corresponding data lineage, and to construct a second database based on the collected historical business data; a joint knowledge vector determination module, configured to determine a plurality of data lineages corresponding to the metadata in the first database, and determine a plurality of joint knowledge vectors based on the metadata and the corresponding data lineages; a first matching module, configured to match a first query vector with the plurality of joint knowledge vectors, and determine a target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to a first query text of a terminal device, and the first query text is natural language text information; An SQL language instruction generation module, configured to generate a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; as well as The first service data returning module is used to query the second database using the first SQL language instruction, obtain and return the corresponding first service data to the terminal device, wherein The SQL language instruction generation module includes: a context information determination module, used to determine context information corresponding to the first query text, wherein the context information is used to indicate a target joint knowledge text corresponding to the target joint knowledge vector; An enhanced information generation module, used to combine the first query text with corresponding context information and generate corresponding enhanced information; a second query text generating module, configured to fill the enhanced information into a preset prompt template and generate a corresponding second query text; and The SQL language instruction generation submodule is used to input the second query text into the generative language model and generate the first SQL language instruction.

9. A query device based on natural language and data knowledge, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Based on the collected metadata and the corresponding data lineage, a first database is constructed, and based on the collected historical business data, a second database is constructed; Determining a plurality of data lineages corresponding to the metadata in the first database, and determining a plurality of joint knowledge vectors based on the metadata and the corresponding respective data lineages; Matching a first query vector with the multiple joint knowledge vectors, and determining a target joint knowledge vector corresponding to the first query vector, wherein the first query vector corresponds to a first query text of a terminal device, and the first query text is natural language text information; Generate a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model; as well as The first SQL language instruction is used to query the second database, obtain and return the corresponding first business data to the terminal device, wherein The operation of generating a first SQL language instruction based on the target joint knowledge vector and a pre-trained generative language model includes: Determining context information corresponding to the first query text, wherein the context information is used to indicate a target joint knowledge text corresponding to the target joint knowledge vector; combining the first query text with corresponding context information and generating corresponding enhanced information; Filling the enhanced information into a preset prompt template and generating a corresponding second query text; and The second query text is input into the generative language model, and the first SQL language instruction is generated.

Images