Information processing method and device, equipment, storage medium and program product

Abstract

The invention discloses an information processing method and device, equipment, a storage medium and a program product, and belongs to the technical field of data processing. The method comprises the steps of obtaining query text information corresponding to a query request in response to the query request; converting the query text information into a query semantic vector; determining a target table vector matched with the query semantic vector from a preset table vector set; according to the target table vector and a preset association relationship between a table vector and table information, target table information is obtained from a database, and the target table information comprises an association table identifier of the target table; obtaining association table information from a database according to the association table identifier; creating a query context object based on the target table information and the association table information so as to generate a target query statement according to the query context object; and executing a query operation in a database according to the target query statement to obtain a query result corresponding to the query request. According to the embodiment of the invention, the database query efficiency and accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and particularly relates to an information processing method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] Under the background of current high-speed development of informatization, the amount of data presents explosive growth, and enterprises are increasingly dependent on data. However, the traditional database query technology faces severe efficiency and accuracy challenges when processing massive data, especially for complex and variable business query requirements, its limitations are increasingly obvious.

[0003] At present, the traditional database query mostly adopts the way of keyword search, which has inherent defects in accurately capturing user query intention, often leading to deviation between query results and actual user needs. At the same time, the rapid iteration and expansion of business make the management complexity of data and metadata rise sharply, which puts forward higher requirements on the efficiency and accuracy of data query technology. SUMMARY

[0004] The embodiments of the present application provide an information processing method, device, equipment and computer storage medium, which can improve the efficiency and accuracy of database query.

[0005] In a first aspect, the embodiments of the present application provide an information processing method, comprising:

[0006] In response to a query request, obtaining query text information corresponding to the query request;

[0007] Converting the query text information into a query semantic vector;

[0008] Determining a target table vector matched with the query semantic vector from a preset table vector set;

[0009] According to the target table vector and a preset association relationship between the table vector and table information, obtaining target table information from a database, the target table information including an associated table identifier of the target table;

[0010] According to the associated table identifier, obtaining associated table information from the database;

[0011] Creating a query context object based on the target table information and the associated table information, so as to generate a target query statement according to the query context object;

[0012] According to the target query statement, performing a query operation in the database to obtain a query result corresponding to the query request.

[0013] In a possible implementation manner, the table vector set includes a plurality of table vectors corresponding to a plurality of table information.

[0014] determining a target table vector matching the query semantic vector from a preset table vector set, comprising:

[0015] respectively calculating similarities between the plurality of table vectors and the query semantic vector;

[0016] determining the target table vector according to the similarities between the plurality of table vectors and the query semantic vector.

[0017] In a possible implementation, before determining the target table vector matching the query semantic vector from the preset table vector set, the method further comprises:

[0018] obtaining a plurality of table information from a database;

[0019] respectively extracting preset field information from the table information for each table information;

[0020] performing semantic vector conversion on the preset field information corresponding to each table information to obtain a first table vector corresponding to each table information;

[0021] storing the first table vector corresponding to each table information to a table vector set.

[0022] In a possible implementation, after respectively extracting the preset field information from the table information for each table information, the method further comprises:

[0023] determining target field information from the preset field information;

[0024] performing semantic extension on the target field information to obtain extended field information;

[0025] performing semantic vector conversion on the extended field information corresponding to each table information to obtain a second table vector corresponding to each table information;

[0026] storing the second table vector corresponding to each table information to the table vector set.

[0027] In a possible implementation, obtaining a plurality of table information from a database comprises:

[0028] obtaining metadata of a plurality of data tables and table relationship information of the plurality of data tables from the database;

[0029] performing normalization processing on the metadata of the plurality of data tables and the table relationship information of the plurality of data tables to obtain the plurality of table information.

[0030] In a possible implementation, after storing the first table vector corresponding to each table information to the table vector set, the method further comprises:

[0031] In a case where table information of the database is updated, at least one updated table information is acquired, the table information update including newly added table information and / or at least one table information changed in the plurality of table information;

[0032] For each updated table information, preset field information is extracted from the updated table information respectively;

[0033] The preset field information corresponding to each updated table information is subjected to semantic vector conversion to obtain a first table vector corresponding to each updated table information;

[0034] The first table vector corresponding to each updated table information is stored in a table vector set.

[0035] In a second aspect, an embodiment of the present application provides an information processing apparatus, comprising:

[0036] An acquisition module is configured to acquire query text information corresponding to a query request in response to the query request;

[0037] A conversion module is configured to convert the query text information into a query semantic vector;

[0038] A determination module is configured to determine a target table vector matched with the query semantic vector from a preset table vector set;

[0039] The acquisition module is further configured to acquire target table information from the database according to the target table vector and a preset association relationship between the table vector and table information, the target table information including an associated table identifier of the target table;

[0040] The acquisition module is further configured to acquire associated table information from the database according to the associated table identifier;

[0041] A creation module is configured to create a query context object based on the target table information and the associated table information, so as to generate a target query statement according to the query context object;

[0042] A query module is configured to perform a query operation in the database according to the target query statement, to obtain a query result corresponding to the query request.

[0043] In a third aspect, an embodiment of the present application provides an information processing device, comprising:

[0044] A processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the information processing method of any one of the above.

[0045] In a fourth aspect, an embodiment of the present application provides a computer storage medium, a computer readable storage medium storing computer program instructions, the computer program instructions being executed by a processor to implement the information processing method of any one of the above.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product. Instructions in the computer program product, when executed by a processor of an electronic device, enable the electronic device to perform the information processing method of any one of the above.

[0047] The information processing method, apparatus, device and computer storage medium provided in the embodiments of the present application can respond to a query request, acquire query text information corresponding to the query request, and convert the query text information into a query semantic vector. Then, a target table vector matching the query semantic vector is determined from a preset table vector set. Compared with a keyword-based search technology, the query based on the semantic vector can capture the implied semantics in the query request, thereby realizing deep understanding of the user query intention and providing more relevant and accurate search results. Subsequently, the target table information can be acquired from a database according to the target table vector and an association relationship between the preset table vector and table information. The target table information includes an associated table identifier of the target table. Then, the associated table information is acquired from the database according to the associated table identifier. The query context object is created based on the target table information and the associated table information, and is used to generate a target query statement according to the query context object. In this way, the vector query and the traditional structured query language (SQL) can be combined to quickly locate the target table, and the relationship structure between tables is used to further refine the query statement, thereby improving the query efficiency and enhancing the accuracy of information and the reliability of the query. Subsequently, a query operation is performed in the database according to the target query statement, and a query result corresponding to the query request is obtained. In this way, the advantages of the vector search and the structured database query can be combined to optimize the data retrieval process and provide more accurate data association analysis, thereby improving the accuracy of the query result. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced. Those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0049] Figure 1 is a structural schematic diagram of an information processing architecture provided by an embodiment of the present application;

[0050] Figure 2 is a flowchart of an information processing method provided by another embodiment of the present application;

[0051] Figure 3 is a structural schematic diagram of an information processing apparatus provided by another embodiment of the present application;

[0052] Figure 4Fig. 1 is a schematic diagram of an information processing device according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] The features and exemplary embodiments of the various aspects of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not meant to limit the present application. The present application can be implemented without some of the specific details of the embodiments described herein. The description of the embodiments is merely provided to give a better understanding of the present application through showing examples of the present application.

[0054] It should be noted that the terms such as first and second, etc., are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Also, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the elements defined by the statement "include" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0055] It should be noted that the acquisition, storage, use and processing of data in the embodiments of the present application comply with the relevant provisions of national laws and regulations.

[0056] It should be noted that in the embodiments of the present application, some software, components, models and other industry existing solutions may be mentioned, which should be considered as exemplary, and the purpose is only to illustrate the feasibility of the implementation of the technical solutions of the present application, but does not mean that the applicant has or will necessarily use the solution.

[0057] Currently, traditional database queries mostly use keyword search. Specifically, keyword search can be implemented through a keyword-based search engine. In the structure of the keyword-based search engine, a reverse index is created by extracting text data from the database. The reverse index records in which documents or data records each keyword appears. After the user submits a query, the search engine can find data records containing all keywords according to the reverse index. This process can include Boolean operations (such as AND, OR) to handle queries containing multiple keywords. The result sorting can be implemented based on a simple model such as term frequency-inverse document frequency (TF-IDF) and the like.

[0058] In the technical solution of realizing database query through a keyword-based search engine, most conventional database management systems (DBSMs) can provide full-text search functions. Based on the full-text search functions, keyword matching can be realized. When a user submits a query containing one or more keywords, the system can find records matching the keywords in the database index.

[0059] In some related technologies, conventional database systems involve using a query optimizer to select the best query execution plan by analyzing the statistical information of the query. However, this optimization method can only improve the query efficiency and cannot improve the accuracy of the query.

[0060] In some other related technologies, conventional database systems also involve using data structures such as B trees, hash tables, and the like to index data to speed up queries. However, these indexes are usually based on the literal values of the data rather than the semantic content thereof, and still cannot accurately identify the query intent of the user.

[0061] In summary, conventional database search technologies are mainly based on keyword matching and lack deep semantic understanding of query statements, and usually cannot accurately understand the query intent of the user, especially when facing complex or ambiguous queries. As a result, the query results often deviate from the actual needs of the user, reducing the accuracy of the query and the user experience. In addition, conventional index technologies such as B trees and hash tables mainly rely on static data structures, and any modification or update of the database structure may require manual intervention to ensure the accuracy of the index and query logic, which not only increases the burden of system operation and maintenance, but also increases the risk of errors, especially in a big data environment, the maintenance cost and complexity are multiplied. Frequently updated data may also require index reconstruction, which is particularly time-consuming in a large-scale database environment. Moreover, most keyword-based search engines use relatively simple models such as TF-IDF to sort search results, and these models cannot effectively reflect complex user needs and lack in-depth evaluation of the quality and relevance of the results. The result sorting may also be inaccurate, and the user needs to manually filter and evaluate a large number of results, further reducing the efficiency of the query and the user experience.

[0062] To solve the problems in the prior art, the embodiments of the present application provide an information processing method, device, equipment and computer storage medium.

[0063] The information processing method provided by the embodiments of the present application can be implemented based on the information processing architecture as shown in the accompanying drawings. Figure 1 The information processing architecture as shown in the accompanying drawings will be briefly introduced first. Figure 1 The information processing architecture as shown in the accompanying drawings will be briefly introduced first.

[0064] As shown in FIG. 1, the information processing architecture includes a data source 101, a data processing layer 102, a query processing layer 103, a result processing layer 104, and a user interface layer 105.Figure 1 As shown, the information processing architecture can include a question query module 101, a vectorization module 102, a vector database search engine 103, a database 104, an external data source 105, a semantic information generation module 106, a manual information maintenance module 107, a data listener 108, a query result generation module 109, and a return information module 110.

[0065] The question query module 101 can be used to obtain query requests submitted by users from a front-end application. These query requests are generally natural language queries and can involve data retrieval, report generation, or data analysis requirements.

[0066] The vectorization module 102 can be used to vectorize natural language in query requests and field information of data tables in the database.

[0067] The vector database search engine 103 can be used to store vector information and perform vector matching with query requests to find the most relevant data. The vector database search engine can be based on any vector search engine, such as Elasticsearch, to quickly retrieve information that is semantically related to user queries from large-scale data sets. The vector database search engine can also be based on a vector database, such as Faiss, Annoy, or Milvus, etc. For applications with extremely high real-time requirements, in-memory computing and storage vectors can also be considered to implement the functions of the vector database search engine.

[0068] The database 104 can be used to store table information and other content.

[0069] The external data source 105 can be used to manage table information of external databases to be processed.

[0070] The semantic information generation module 106 can use a large language model (LLM) to perform semantic analysis and extended description generation on fields related to table information in the database, to improve the queryability and relevance of the data.

[0071] The manual information maintenance module 107 can provide a maintenance portal for database administrators to maintain business-intensive table fields, table relationships, and other content.

[0072] The data listener 108 can continuously monitor updates to external data sources and automatically trigger synchronization updates to internal databases to ensure data consistency and up-to-date status.

[0073] The query result generation module 109 can be used to match relevant table information and generate corresponding query results based on matched vectors.

[0074] The return information module 110 can return the matched table information content or the query result.

[0075] Before performing the information processing method based on the information processing architecture, an offline vector index needs to be created. The following briefly introduces the process of creating the offline vector index. For example, the process of creating the offline vector index can include data collection, vectorization, index update and maintenance setting, and query request matching rule setting.

[0076] The data collection can include extracting metadata of all data tables from the data warehouse. For example, the metadata can include a plurality of field information, such as table name, table annotation information, column name, column type, and column annotation information. In one example, the relationship between tables can be configured by a database administrator. The data collection can also include obtaining table relationship information of each data table.

[0077] After extracting the metadata of all data tables from the data warehouse, the field information can be standardized. For example, the table annotation information and the column annotation information can be standardized to obtain corresponding table description information and table field description. The standardization can be achieved according to methods known in the art, which will not be described here. For example, the standardization can include normalizing synonymous expressions or common variants to a standard form.

[0078] After extracting the metadata of all data tables from the data warehouse, the fields with limited value domain and possibly suitable for defining as enumeration types in the table can be created into new table field information based on all unique values of the column in the database. The new table field information can be referred to as table field value. The fields with limited value domain and possibly suitable for defining as enumeration types can include fields with a number of all unique values of the column in the database less than a preset threshold (for example, less than 20).

[0079] Then, the field information including table name, table relationship, table description information, table field name, table field value, and table field description can be summarized.

[0080] In one example, for the core field information of some core business tables, an extension description process can also be performed. Specifically, the LLM can generate extension field information of the table extension description, table field extension description, table core field name, table label, table field, and the like in multiple different angles under the business field. Subsequently, the previously aggregated field information and the extension field information can be persisted in the database. In an example, the LLM can use Bidirectional Encoder Representations from Transformers (BERT) or Generative Pre-trained Transformer (GPT) from the transformer. The LLM can also use other pre-trained transformer models such as RoBERTa, XLNet, or DistilBERT, and the like. The LLM can also use other custom trained models, for example, models trained for specific data sets and business needs to more effectively capture specific business semantics and terminology.

[0081] Vectorization can include table structure vectorization. Specifically, the table name, table relationship, table description information, table field name, table field value, and table field description can be obtained from the database, preprocessed by a text preprocessing function, and organized into a JavaScript Object Notation (JSON) format. Subsequently, a Natural Language Processing (NLP) technique can be used, for example, using the bert-base-multilingual-cased pre-trained model in BERT with parameters set, and the maximum length of the token sequence after word segmentation can be set to 512, for example, returning a PyTorch tensor, generating a semantic vector for each table corresponding to the above field information, obtaining a first table vector corresponding to each table information, and storing the vector Identity document (ID) and table information ID in the database. In one example, the table name, table field name, and table core field name, table extension description, table field extension description, table label, table field, and the like can also be vectorized in the same way, obtaining a second table vector corresponding to each table information, and storing the vector ID and table information ID in the database.

[0082] It can be understood that according to the above embodiments, each first table vector can uniquely correspond to one table information, and each second table vector can uniquely correspond to one table information. For one table information, it can correspond to at least one first table vector and at least one second table vector.

[0083] In one example, before the table structure vectorization is performed, a text preprocessing function can be defined. The embodiments of the present application do not limit the specific composition of the text preprocessing function. For example, the text preprocessing function can be obtained by combining a character set standardization function, a noise removal function, a word segmentation processing function, a database word segmentation function, and a stop word removal function. The character set standardization function can be used to ensure that the text data is converted to a unified character encoding (UTF-8); the noise removal function can be used to remove special characters, punctuation marks, etc. using a regular expression; the word segmentation processing function can be used to perform Chinese word segmentation using jieba segmentation, which facilitates subsequent replacement; the database word segmentation function can be used to perform word segmentation processing on the camel case, underscore, etc. in the field information; and the stop word removal function can be used to remove stop words in the text according to a stop word list prepared in advance, wherein the stop word list can be obtained from an open source.

[0084] The index update and maintenance setting can specifically include using an application programming interface (API) hook to listen to data changes in real time. When a user calls an API interface to update table information, the vector index can be triggered to update. Specifically, the newly added or updated data can be subjected to a re-vectorization process.

[0085] The query request matching rule setting can include the following settings for the user query process: the user's query information is preprocessed by a text preprocessing function. The preprocessed text is converted into a query semantic vector by BERT, the similarity between the query semantic vector and the first table vector and the second table vector stored in the database is calculated, and the top N most relevant vector indexes are obtained, N being a positive integer. According to the indexes, the relevant table information and table relationships in the database are queried. These information is returned to obtain the table information of the associated table through the table relationship. The table information and the associated table information are used as contexts to generate SQL corresponding to each vector index, and the SQL is used to perform a query operation in the database to return the query results corresponding to each vector index. The query results corresponding to each vector index can be sorted in descending order of the aforementioned similarity.

[0086] In one example, the above-mentioned table information of the associated table obtained through the table relationship can include the table information of the associated table obtained through the foreign key of the table. In one example, the table information of the associated table can also be obtained by using a graph database query technology. In one example, a machine learning algorithm can be used to predict and verify the potential relationship between the data to obtain the associated information in other tables related to the table information. In this way, the implicit relationship between the data can be automatically discovered and utilized without explicit foreign keys.

[0087] According to the above process, the offline vector index is created, and the semantic vectorization of metadata is implemented by using a deep learning model such as BERT, realizing the deep understanding of the user query intention. In this way, the problem of inaccurate query results caused by relying only on keyword matching can be effectively alleviated, especially when dealing with complex queries containing polysemous words or requiring deep business understanding. In addition, the offline vector index introduces a dynamic vector index mechanism, allowing real-time updating of the vector representation of the data table, thereby reflecting any changes in the database in real time. In this way, it is beneficial to maintain the query accuracy and response speed of large-scale databases. Compared with the traditional index updating method (such as completely rebuilding the index), this method greatly reduces the maintenance cost and improves the running efficiency of the system, especially suitable for application environments where data frequently changes. Further, the offline vector index ensures that the index is always synchronized with the latest state of the database through an automated updating mechanism, reducing query errors caused by data delays or asynchronization. In this way, the need for manual intervention can be reduced, the operating cost can be reduced, and the reliability and stability of the system can be improved.

[0088] In addition, the vector-based search method combines a similarity calculation algorithm to more accurately rank the query results. This method can provide ranking based on the deep semantics of the content, which is much better than traditional TF-IDF and other models in evaluating the relevance of query results. In this way, the accuracy of result ranking is greatly improved, and the user experience when searching for specific data is improved.

[0089] The information processing method provided by the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0090] Figure 2 A flowchart of an information processing method provided by an embodiment of the present application is shown. As shown in Figure 2 The method can include the following steps S210 to S270.

[0091] S210, in response to a query request, obtaining query text information corresponding to the query request.

[0092] In step S210, the query request can include a natural language query request generated in response to the user's input operation on the front-end interface.

[0093] S220, converting the query text information into a query semantic vector.

[0094] In step S220, the query text information can be converted into a query semantic vector by using an NLP technology. In one example, keywords can be extracted from the query text information, and the keywords can be converted to obtain the query semantic vector. In another example, keywords can also be extracted from the query text information, the keywords can be semantically expanded, and the keywords and the semantic expansion results can be converted to obtain the query semantic vector. In this way, the accuracy of user query intent recognition can be further improved, thereby improving the accuracy of the query result.

[0095] In step S230, a target table vector matched with the query semantic vector is determined from a preset table vector set.

[0096] In step S230, the preset table vector set can include a plurality of table vectors generated based on a plurality of table information corresponding to a plurality of data tables in a database. The table vector set can be managed by a vector database search engine.

[0097] In step S240, target table information including an associated table identifier of the target table is obtained from the database according to the target table vector and an association relationship between the preset table vector and the table information.

[0098] In step S250, associated table information is obtained from the database according to the associated table identifier.

[0099] In step S260, a query context object is created based on the target table information and the associated table information, and is used to generate a target query statement according to the query context object.

[0100] In step S260, the target query statement can include a SQL corresponding to the query request. Based on the target table information and the associated table information, the target query statement can be generated, which can include creating a query context object based on the target table information and the associated table information. According to the query context object, a SQL query statement can be constructed which can fully utilize the context information. For example, according to the business logic and the query requirement, the association relationship between the relevant table information and the tables in the target table information and the associated table information can be determined as the context information, and the query context object can be created. Then, the table information in the query context object can be traversed, and a basic SELECT statement can be generated for each table. According to the foreign key relationship between the target table and the associated table, a JOIN clause in the SQL can be constructed. For the case where there are a plurality of associated tables, nested queries or multiple JOINs can also be used to associate all the tables. According to the query field or other business logic, a filtering condition such as a WHERE clause can be added to the SQL. Optionally, a sorting and grouping clause can also be added as needed. Then, a SQL function can be selected to generate the target SQL.

[0101] S270, performing a query operation in the database according to the target query statement to obtain a query result corresponding to the query request.

[0102] The information processing method of the embodiments of the present application can respond to a query request, acquire query text information corresponding to the query request, and convert the query text information into a query semantic vector. Then, a target table vector matching the query semantic vector is determined from a preset table vector set. Compared with keyword-based search technology, the query through the semantic vector can capture the implied semantics in the query request, thereby realizing deep understanding of the user query intention and providing more relevant and accurate search results. Subsequently, the target table information can be acquired from the database according to the target table vector and the association relationship between the preset table vector and the table information. The target table information includes an associated table identifier of the target table. Then, the associated table information is acquired from the database according to the associated table identifier. The query context object is created based on the target table information and the associated table information, and used to generate the target query statement according to the query context object. In this way, the target table can be quickly located by combining the vector query and the traditional SQL, and the query statement can be further refined by using the relationship structure between the tables, thereby improving the query efficiency while enhancing the accuracy of the information and the reliability of the query. Subsequently, a query operation is performed in the database according to the target query statement to obtain a query result corresponding to the query request. In this way, the advantages of the vector search and the structured database query can be combined to optimize the data retrieval process and provide more accurate data association analysis, thereby improving the accuracy of the query result.

[0103] In one embodiment, the table vector set can include a plurality of table vectors corresponding to a plurality of table information.

[0104] The target table vector matching the query semantic vector can be determined from the preset table vector set, and the determination can include:

[0105] The similarity between each table vector and the query semantic vector is calculated.

[0106] The target table vector is determined according to the similarity between each table vector and the query semantic vector.

[0107] In the above embodiments, the similarity can include any similarity suitable for the semantic vector, for example, can include cosine similarity, Euclidean distance, Manhattan distance, Pearson correlation coefficient, Jaccard similarity, or semantic similarity measure (such as path similarity in WordNet, Wu-Palmer similarity, etc.). The present application does not make any limitation in this regard.

[0108] In the above embodiments, the number of target table vectors can be one or more than two. When the number of target table vectors is more than two, the target table vectors are determined according to the similarity between each table vector and the query semantic vector, which can specifically include: determining the table vectors with a similarity greater than a preset threshold as the target table vectors; or can include: sorting each table vector in descending order of similarity; and determining the top N table vectors as the target table vectors, N being a positive integer. Accordingly, in one example, when the number of target table vectors is more than two, for each target table vector, the target table information corresponding to the target table vector can also be obtained from the database according to the preset association relationship between the table vector and the table information, and the associated table information corresponding to the target table vector can be obtained from the database according to the associated table identifier in the target table information. Subsequently, the query context object corresponding to the target table vector is created based on the target table information and the associated table information corresponding to the target table vector, and the target query statement corresponding to the target table vector is generated according to the query context object. Then, the query operation is performed in the database according to the target query statement corresponding to the target table vector, and the query result corresponding to the target table vector is obtained. In this way, after obtaining the query results corresponding to each target table vector, the query results corresponding to each target table vector can also be sorted according to the similarity between the target table vector and the query semantic vector. In this way, the multiple query results can be sorted based on deep semantic, so that the sorting of the query results is more consistent with the user's intention, thereby facilitating the improvement of the user experience in searching for specific data.

[0109] According to the above embodiments, the target table vector can be determined according to the similarity between each table vector and the query semantic vector, thereby realizing the screening of table information. In this way, the target table information that is more consistent with the user's query intention can be determined, and the query result that is more consistent with the user's query intention can be obtained. In this way, it is beneficial to further improve the accuracy of database query, thereby improving the user's query experience.

[0110] In one embodiment, the preset table vector set can include a plurality of first table vectors corresponding to a plurality of table information.

[0111] Before determining the target table vector matching the query semantic vector from the preset table vector set, the method can further include:

[0112] Obtaining a plurality of table information from the database.

[0113] For each table information, the preset field information is extracted from the table information.

[0114] The preset field information corresponding to each table information is subjected to semantic vector conversion to obtain a first table vector corresponding to each table information.

[0115] The first table vector corresponding to each table information is stored in the table vector set.

[0116] In the above embodiments, the preset field information can include field information directly obtained from the table information, for example, can include table name, table relationship, table description information, table field name, table field value and table field description.

[0117] According to the above embodiments, by extracting the preset field information from the table information, and then performing semantic vector conversion. In this way, the original field information in the data table can be vectorized, thereby allowing the search of the table information based on the semantic similarity in the vector space. In this way, the efficiency and accuracy of the database query can be improved.

[0118] In one embodiment, the preset table vector set can further include a plurality of second table vectors corresponding to the plurality of table information.

[0119] For each table information, after extracting the preset field information from the table information, the method can further include:

[0120] Determining target field information from the preset field information.

[0121] Performing semantic expansion on the target field information to obtain expanded field information.

[0122] Performing semantic vector conversion on the expanded field information corresponding to each table information to obtain a second table vector corresponding to each table information.

[0123] Storing the second table vector corresponding to each table information to the table vector set.

[0124] In the above embodiments, the target field information can include field information determined based on business rules, for example, can include core field information of core business tables. The expanded field information can include, for example, table core field name, table expansion description, table field expansion description, table label, table field, and the like. When performing semantic vector conversion on the expanded field information corresponding to each table information, the table name and the table field name can also be used as a set with the above expanded field information for semantic vector conversion.

[0125] In the above embodiments, the semantic expansion can be implemented in various ways. As an example, NLP techniques such as Word Embedding, Semantic Role Labeling (SRL), etc. can be used to perform semantic analysis on the target field and find words or phrases that are semantically similar to it. As another example, machine learning models (such as classifiers, clusterers) or deep learning models (such as neural networks) can also be trained to learn the semantic relationships between the target field and other fields, and expand based on these relationships. For example, a deep learning model can be trained on a large amount of text data to learn the semantic similarity between different words, and then expand the target field based on these similarities. As yet another example, the target field can also be expanded based on predefined rules or templates. The above predefined rules or templates can be based on domain knowledge, expert experience or statistical regularities. As yet another example, a thesaurus or word bank can also be used to find words with the same or similar meanings as the target field and expand them. The semantic expansion can also be implemented based on a combination of two or more of the above examples, but is not limited thereto.

[0126] According to the above embodiments, the target field information is semantically expanded to obtain expanded field information, and then the second table vector corresponding to each table information is obtained based on the expanded field information. In this way, the relatively important field information can be described in multiple angles. In this way, when the user queries the corresponding information through different keywords, the relevant table information can be accurately matched, and further relevant query results can be obtained. Therefore, the accuracy of the query and the user experience can be improved.

[0127] In one embodiment, the preset table vector set can include a plurality of first table vectors and a plurality of second table vectors corresponding to a plurality of table information.

[0128] Determining the target table vector that matches the query semantic vector from the preset table vector set can specifically include:

[0129] Respectively calculating the similarity between each first table vector and each second table vector and the query semantic vector.

[0130] Determining the target table vector according to the similarity between each first table vector and each second table vector and the query semantic vector.

[0131] In the above embodiments, each first table vector can correspond to one table information uniquely, and each second table vector can correspond to one table information uniquely. It should be noted that, although the first table vector and the second table vector can correspond to the same table information, they are compared independently when comparing the similarity. That is, the similarity values of the first table vector and the second table vector of the same table information are not combined or averaged. In an example, when the number of target table vectors is more than two, before obtaining the target table information from the database according to the target table vector and the preset association relationship between the table vector and the table information, the method can further include: performing a deduplication processing on the two or more target table vectors. Specifically, for the multiple table vectors corresponding to the same table information in the two or more target table vectors, only one can be retained as a target vector, so that the risk of repeated query results can be reduced, and the simplicity of the query results can be improved.

[0132] According to the above embodiments, the different semantic vectors corresponding to the respective table information and the query semantic vector can be compared in terms of similarity, and the target table vector can be determined according to the comparison result. In this way, when the user queries the corresponding information through different keywords, the related table information can be matched from multiple angles, so that the risk of missing the query can be reduced. In this way, the accuracy of the query and the user experience can be further improved.

[0133] In an embodiment, obtaining the multiple table information from the database can specifically include:

[0134] Obtaining the metadata of the multiple data tables and the table relationship information of the multiple data tables from the database.

[0135] Performing a normalization processing on the metadata of the multiple data tables and the table relationship information of the multiple data tables to obtain the multiple table information.

[0136] In the above embodiments, the table relationship information can include the table relationship information configured by the database administrator, for example, can include foreign keys, etc.

[0137] According to the above embodiments, the metadata and the table relationship information of the data table can be extracted and normalized. In this way, the form of the obtained table information is more standard and normative, so that a more accurate table vector can be generated, and the accuracy of the database query can be improved.

[0138] In an embodiment, after storing the first table vector corresponding to each table information into the table vector set, the method can further include:

[0139] In the case that the table information of the database is updated, at least one updated table information is obtained, and the table information update includes adding new table information and / or changing at least one of the multiple table information.

[0140] The preset field information is extracted from each update table information.

[0141] The preset field information corresponding to each update table information is subjected to semantic vector conversion to obtain a first table vector corresponding to each update table information.

[0142] The first table vector corresponding to each update table information is stored in a table vector set.

[0143] In an embodiment, the target field information in the preset field information corresponding to each update table information can also be subjected to semantic expansion to obtain expanded field information, and a second table vector corresponding to each update table information is further generated. The generation process of the second table vector has been described above and will not be repeated here.

[0144] According to the above embodiments, when information in the database is updated, the newly added or updated data can be subjected to a re-vectorization process. In this way, the update vector index can be triggered to realize dynamic updating of the index. In this way, the maintenance cost can be reduced and the operation efficiency of the system can be improved.

[0145] Based on the information processing method provided in the above embodiments, the application also provides a specific implementation of an information processing device. Please refer to the following embodiments.

[0146] Firstly, referring to Figure 3 The information processing device 300 provided in the embodiments of the application includes a first acquisition module 301, a first conversion module 302, a first determination module 303, a creation module 304, and a query module 305.

[0147] The first acquisition module 301 is configured to acquire query text information corresponding to a query request in response to the query request.

[0148] The first conversion module 302 is configured to convert the query text information into a query semantic vector.

[0149] The first determination module 303 is configured to determine a target table vector matched with the query semantic vector from a preset table vector set.

[0150] The first acquisition module 301 is further configured to acquire target table information from a database according to the target table vector and a preset association relationship between the table vector and table information, and the target table information includes an associated table identifier of the target table.

[0151] The first acquisition module 301 is further configured to acquire associated table information from the database according to the associated table identifier.

[0152] The creation module 304 is configured to create a query context object based on the target table information and the associated table information, so as to generate a target query statement according to the query context object.

[0153] The query module 305 is configured to perform a query operation in the database according to the target query statement, to obtain a query result corresponding to the query request.

[0154] The information processing apparatus provided in the embodiments of the present application can respond to a query request, acquire query text information corresponding to the query request, and convert the query text information into a query semantic vector. Then, a target table vector matching the query semantic vector is determined from a preset table vector set. Compared with a keyword-based search technology, the query by means of the semantic vector can capture the implied semantics in the query request, so as to realize deep understanding of the user query intention and provide more relevant and accurate search results. Subsequently, the target table information can be acquired from the database according to the target table vector and an association relationship between the preset table vector and table information. The target table information includes an associated table identifier of the target table. Then, the associated table information is acquired from the database according to the associated table identifier. The query context object is created based on the target table information and the associated table information, so as to generate the target query statement according to the query context object. In this way, the target table can be quickly located by combining the vector query and the traditional SQL, and the query statement can be further refined by using the relationship structure between tables, so as to improve the query efficiency while enhancing the accuracy of information and the reliability of the query. Subsequently, a query operation is performed in the database according to the target query statement, to obtain a query result corresponding to the query request. In this way, the advantages of the vector search and the structured database query can be combined, the data retrieval process can be optimized, and more accurate data association analysis can be provided, so as to improve the accuracy of the query result.

[0155] In one embodiment, the table vector set can include a plurality of table vectors corresponding to a plurality of table information.

[0156] The first determination module is configured to determine, from the preset table vector set, a target table vector matching the query semantic vector, and specifically can include:

[0157] The calculation submodule is configured to respectively calculate the similarity between the plurality of table vectors and the query semantic vector.

[0158] The determination submodule is configured to determine the target table vector according to the similarity between each table vector and the query semantic vector.

[0159] In one embodiment, the apparatus can further include:

[0160] The second acquisition module is configured to acquire, from the database, a plurality of table information before determining, from the preset table vector set, a target table vector matching the query semantic vector.

[0161] The first extraction module is configured to respectively extract, for each table information, preset field information from the table information.

[0162] The second conversion module is configured to perform semantic vector conversion on the preset field information corresponding to each table information, to obtain a first table vector corresponding to each table information.

[0163] The first storage module is configured to store the first table vector corresponding to each table information into the table vector set.

[0164] In an embodiment, the apparatus can further include:

[0165] The second determination module is configured to determine, for each table information, target field information from the preset field information after extracting the preset field information from the table information.

[0166] The expansion module is configured to perform semantic expansion on the target field information, to obtain expanded field information.

[0167] The third conversion module is configured to perform semantic vector conversion on the expanded field information corresponding to each table information, to obtain a second table vector corresponding to each table information.

[0168] The second storage module is configured to store the second table vector corresponding to each table information into the table vector set.

[0169] In an embodiment, the second acquisition module is configured to acquire the plurality of table information from the database, and can specifically include:

[0170] The acquisition submodule is configured to acquire metadata of the plurality of data tables and table relationship information of the plurality of data tables from the database.

[0171] The processing submodule is configured to perform normalization processing on the metadata of the plurality of data tables and the table relationship information of the plurality of data tables, to obtain the plurality of table information.

[0172] In an embodiment, the apparatus can further include:

[0173] The third acquisition module is configured to, after storing the first table vector corresponding to each table information into the table vector set, acquire at least one updated table information in a case where table information update occurs in the database, the table information update including adding new table information and / or changing at least one table information in the plurality of table information.

[0174] The second extraction module is configured to extract, for each updated table information, preset field information from the updated table information.

[0175] The fourth conversion module is configured to perform semantic vector conversion on the preset field information corresponding to each updated table information, to obtain a first table vector corresponding to each updated table information.

[0176] The third storage module is configured to store the first table vector corresponding to each updated table information into the table vector set.

[0177] The various modules of the information processing device provided by the embodiments of the present application can realize Figure 2 The various steps of the information processing method provided by the embodiments of the present application provide the functions of the steps and achieve the corresponding technical effects. For brevity, the details are not described herein.

[0178] Figure 4 A hardware structure schematic diagram of the information processing device provided by the embodiments of the present application is shown.

[0179] The information processing device can include a processor 401 and a memory 402 storing computer program instructions.

[0180] Specifically, the processor 401 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0181] The memory 402 can include a mass storage for data or instructions. By way of example and not limitation, the memory 402 can include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 402 can include removable or non-removable (or fixed) media. Where appropriate, the memory 402 can be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, the memory 402 is non-volatile solid-state memory.

[0182] The memory can include read-only memory (ROM), random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions that, when executed (e.g., by one or more processors), are operable to perform the operations described with reference to the methods according to an aspect of the present disclosure.

[0183] The processor 401 realizes any one of the information processing methods in the above embodiments by reading and executing the computer program instructions stored in the memory 402.

[0184] In one example, the information processing device can further include a communication interface 403 and a bus 410. Among them, as Figure 4As shown, the processor 401, the memory 402, and the communication interface 403 are connected by the bus 410 and accomplish communication between each other.

[0185] The communication interface 403 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.

[0186] The bus 410 includes hardware, software or both to couple components of the information processing device to each other. By way of example, and not limitation, the bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or combination of two or more of these. Where suitable, the bus 410 can include one or more buses. Although specific buses are described and shown in the embodiments of the present application, the present application contemplates any suitable bus or interconnect.

[0187] The information processing device can execute the information processing method in the embodiments of the present application, thereby realizing the information processing method and device described in combination Figure 2 and Figure 3 The information processing method in the above embodiments, the embodiments of the present application can provide a computer storage medium to realize. The computer storage medium has computer program instructions stored thereon; the computer program instructions are executed by the processor to realize the information processing method in any one of the above embodiments.

[0188] The embodiments of the present application also provide a computer program product, comprising a computer program, the computer program is executed by the processor to realize the information processing method in any one of the above embodiments.

[0189] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0190] The functions noted in the description of the structural block diagrams above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium, or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. A "machine-readable medium" includes any medium that can store or transport information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. The code segments can be downloaded via computer networks such as the Internet, intranets, and the like.

[0191] It is also important to note that the examples mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the steps mentioned above, that is, the steps can be performed in the order mentioned in the examples, or in an order different from the examples, or several steps can be performed simultaneously.

[0192] The computer program instructions can also be loaded onto a computer, other programmable information processing apparatus, or other processing devices to cause a series of operational steps to be performed on the computer, other programmable information processing apparatus, or other processing devices to generate a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer program instructions can also be stored in a computer readable medium that can direct a computer, other programmable information processing apparatus, or other processing devices to operate in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0193] The above is merely a specific implementation of the present application. As can be clearly understood by a person skilled in the art from the above description, for the convenience and brevity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described herein again. It should be understood that the protection scope of the present application is not limited in this way, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed in the present application, and these modifications or replacements should be covered in the protection scope of the present application.

Claims

1. An information processing method characterized by comprising: The method comprises the following steps: in response to a query request, obtaining query text information corresponding to the query request; convert the query text information into a query semantic vector; determine the target table vector matched with the query semantic vector from the preset table vector set; According to the target table vector and the association relationship between the preset table vector and the table information, obtain the target table information from the database, and the target table information includes the association table identifier of the target table; According to the association table identifier, obtain the association table information from the database; Create a query context object based on the target table information and the association table information, and generate a target query statement based on the query context object; According to the target query statement, execute the query operation in the database to obtain the query result corresponding to the query request.

2. The information processing method according to claim 1, characterized by, The table vector set includes a plurality of table vectors corresponding to a plurality of table information; The method further comprises the following steps before determining the target table vector matched with the query semantic vector from the preset table vector set: Obtain a plurality of table information from the database; For each table information, extract the preset field information from the table information respectively; 3. The information processing method according to claim 1, characterized by, Perform semantic vector conversion on the preset field information corresponding to each table information to obtain a first table vector corresponding to each table information; Store the first table vector corresponding to each table information into the table vector set. The method further comprises the following steps after extracting the preset field information from the table information respectively: Determine the target field information from the preset field information; Perform semantic expansion on the target field information to obtain expanded field information; 4. The information processing method according to claim 3, characterized by, Perform semantic vector conversion on the expanded field information corresponding to each table information to obtain a second table vector corresponding to each table information; Store the second table vector corresponding to each table information into the table vector set. Obtaining the plurality of table information from the database comprises: Obtain the metadata of a plurality of data tables and the table relationship information of the plurality of data tables from the database; Normalize the metadata of the plurality of data tables and the table relationship information of the plurality of data tables to obtain the plurality of table information.

5. The information processing method according to claim 3, characterized by, The method further comprises the following steps after storing the first table vector corresponding to each table information into the table vector set: In the case that the table information in the database is updated, obtain at least one updated table information, and the table information update includes adding new table information and / or changing at least one of the plurality of table information; For each updated table information, extract the preset field information from the updated table information respectively; 6. The information processing method according to claim 3, characterized by, Perform semantic vector conversion on the preset field information corresponding to each updated table information to obtain a first table vector corresponding to each updated table information; Store the first table vector corresponding to each updated table information into the table vector set. The method comprises the following steps: ​ ​ 7. An information processing apparatus, characterized by comprising: ​ An obtaining module is configured to obtain query text information corresponding to a query request in response to the query request; A converting module is configured to convert the query text information into a query semantic vector; A determining module is configured to determine a target table vector matched with the query semantic vector from a preset table vector set; The obtaining module is further configured to obtain target table information from a database according to the target table vector and an association relationship between preset table vectors and table information, the target table information including an associated table identifier of the target table; The obtaining module is further configured to obtain associated table information from the database according to the associated table identifier; A creating module is configured to create a query context object based on the target table information and the associated table information, so as to generate a target query statement according to the query context object; A querying module is configured to perform a query operation in the database according to the target query statement, and obtain a query result corresponding to the query request.

8. An information processing apparatus, characterized by comprising: The device comprises a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the information processing method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the information processing method of any one of claims 1-6.

10. A computer program product, characterised in that, The instructions in the computer program product are executed by the processor of the electronic device, so that the electronic device can execute the information processing method of any one of claims 1-6.

Images