Industrial data intelligent query method and system based on large language model

Abstract

The present application relates to the field of data processing, and more particularly to an industrial data intelligent query method and system based on a large language model, which comprises the following steps: calculating the contribution degree of each dimension of each data table in each historical query; calculating the preference value of each dimension; calculating the basic score of each data table; calculating the inter-table similarity between the data table and each other data table, weighting and fusing the inter-table similarity by taking the basic score of each other data table as the weight, and combining the frequency of the data table being selected by the target user to calculate the relative importance factor of the data table; multiplying the basic score of the data table by the relative importance factor to obtain the final score of the data table, and arranging all data tables in descending order according to the final score for the target user to query. The recommended result of the present application is deeply adapted to the query habits of individual users, and the joint exposure rate of associated data tables is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing. In particular, it relates to a method and system for intelligent querying industrial data based on a large language model. Background Technology

[0002] Traditional industrial data query methods typically rely on fixed filtering interfaces built into business systems. Production managers must manually locate target data by clicking through menus, filtering by conditions, and selecting fields. Because industrial data involves numerous tables with complex structures and diverse field semantics, and because data silos exist between different business systems, managers often need to switch between multiple system interfaces, resulting in low query efficiency and high learning costs.

[0003] In recent years, some solutions have attempted to introduce large language models to achieve semantic conversion from natural language to database queries. By calculating the semantic similarity between the user's query text and the metadata of the data table (such as table name, field name, table comments, etc.), relevant data tables are automatically recommended.

[0004] However, existing technologies still have the following limitations: they only focus on the instant semantic matching of a single query and a data table. Managers in different positions and with different responsibilities have significantly different focuses on the same type of data, making it difficult to adapt to the true query intent of individual users using a unified semantic matching standard. In industrial data scenarios, business-related tables often need to be jointly queried, but existing technologies lack modeling of inter-table relationships, resulting in fragmented recommendation results that cannot meet users' overall needs for related data, leading to low query efficiency and a poor user experience.

[0005] Therefore, there is a need in this field for an intelligent query method and system for industrial data based on a large language model to solve the problems of low query efficiency and poor user query experience of the above-mentioned query schemes. Summary of the Invention

[0006] To address the technical problems of low query efficiency and poor user query experience in existing query solutions, this invention provides solutions in the following aspects.

[0007] In the first aspect, the intelligent query method for industrial data based on large language models includes:

[0008] A large language model is used to extract semantic vector sequences of the target user's current query text and the text of each dimension of each data table in the database, and the relevance vector between the target user's current query text and each data table is calculated based on the semantic vector sequences.

[0009] Obtain the relevance vector between the target user's historical query text and each data table. For each historical query, calculate the contribution of each dimension of the data table in the historical query based on the relevance vector between the historical query text and each data table. Calculate the correlation between the historical query text and the current query text. Calculate the preference value of each dimension based on the correlation between the historical query text and the current query text and the contribution of each dimension of the data table in each historical query.

[0010] For each data table, calculate the basic score of the data table based on the relevance vector between the data table and the current query text and the preference value of each dimension; calculate the inter-table similarity between the data table and each other data table, use the basic scores of each other data table as weights to perform weighted fusion of the inter-table similarity, and calculate the relative importance factor of the data table by combining the frequency of the data table being selected by the target user.

[0011] Multiply the base score of the data table by the relative importance factor to obtain the final score of the data table. Sort all data tables in descending order of the final score for target users to query.

[0012] Preferably, the calculation of the relevance vector between the target user's current query text and each data table based on the semantic vector sequence includes: taking any dimension of the data table as the target dimension; for each data table, calculating the cosine similarity value between each semantic vector in the semantic vector sequence of the current query text and each semantic vector in the semantic vector sequence of the target dimension text; taking the average of all normalized cosine similarity values ​​as the relevance value between the current query text and the target dimension text; the relevance values ​​corresponding to the text of all dimensions form the relevance vector between the current query text and the data table.

[0013] Preferably, the method for calculating the contribution of each dimension of the data table in the historical query includes: obtaining the first set of data tables selected by the target user and the second set of data tables not selected in the historical query; obtaining the relevance value between the target dimension text of the data table and the text of the historical query from the relevance vector, and calculating the first average value of the relevance value between the target dimension text of each data table in the first set of data tables and the text of the historical query; calculating the second average value of the relevance value between the target dimension text of each data table in the second set of data tables and the text of the historical query; calculating the difference between the first average value and the second average value, and normalizing the difference to obtain the contribution of the target dimension.

[0014] Preferably, the method for calculating the preference value of each dimension includes: for each historical query, using the DTW algorithm to calculate the shortest path distance between the semantic vector sequence of the historical query text and the semantic vector sequence of the current query text, performing negative correlation normalization on the shortest path distance to obtain the relevance; multiplying the relevance by the contribution of the target dimension in the historical query to obtain the first product corresponding to the target dimension in the historical query; summing the first products corresponding to the target dimensions in all historical queries to obtain the preference value of the target dimension.

[0015] Preferably, the method for calculating the basic score of the data table includes: normalizing the preference values ​​of all dimensions of the data table so that the sum of all normalized preference values ​​is 1; for each dimension, multiplying the relevance value of the text of that dimension to the current query text by the normalized preference value of that dimension to obtain the second product corresponding to that dimension; summing the second products corresponding to all dimensions to obtain the basic score of the data table.

[0016] Preferably, the method for calculating the relative importance factor of the data table includes: obtaining the number of times the data table is selected by all users and normalizing it to obtain the normalized number; the relevance vector of the data table and the current query text, together with the normalized number of the data table, constitute an influencing factor sequence; calculating the cosine similarity value between the influencing factor sequences of the data table and each other data table to obtain the inter-table similarity; using the basic score of each other data table as a weight to perform weighted fusion of the corresponding inter-table similarities to obtain a weighted fusion value; calculating the probability that the data table is selected by the target user, multiplying the probability by the weighted fusion value to obtain a third product, and normalizing the third product to obtain the relative importance factor.

[0017] Preferably, the extraction of the semantic vector sequence of the target user's current query text using a large language model includes: performing word segmentation on the current query text to obtain a word sequence of the current query text; and using a pre-trained large language model to extract the semantic vector of each word in the word sequence to obtain the semantic vector sequence of the current query text.

[0018] Preferably, each data table has four dimensions of text: table name, table comment, field name, and field comment.

[0019] Secondly, an intelligent industrial data query system based on a large language model includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the aforementioned intelligent industrial data query method based on a large language model is implemented.

[0020] The present invention has the following effects:

[0021] 1. This invention analyzes the selection behavior of target users in their historical queries, quantifies the differentiated preferences of users for each dimension in the data table, and in the basic score calculation stage, it performs weighted fusion of the semantic relevance vectors of each dimension based on these preferences. This achieves the effect that for the same current query text, different users obtain differentiated basic scores and ranking results in the data table due to their different preferences for each dimension. Compared with the undifferentiated recommendation mode of the existing technology, this invention realizes personalized intelligent query with a thousand faces for a thousand people, and makes the recommendation results deeply adapted to the individual query habits of users.

[0022] 2. This invention constructs a sequence of influencing factors containing a relevance vector and a global selection frequency normalization value, calculates inter-table similarity, and weights and fuses the inter-table similarity based on the basic scores of other data tables. This allows the importance of data tables with high basic scores to be transmitted to related data tables along the inter-table similarity relationship. Even if a data table itself has a low semantic matching degree with the current query text, as long as it has a high similarity with several high basic score tables, it can still obtain a reasonable relative importance factor. This effectively overcomes the fragmentation problem of recommendation results caused by the isolated evaluation of each data table in traditional methods, significantly improves the joint exposure rate of related data tables, meets the overall needs of association analysis in industrial scenarios, and improves query efficiency and user experience. Attached Figure Description

[0023] Figure 1 This is a flowchart of steps S1-S6 in an industrial data intelligent query method based on a large language model according to an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0025] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0026] Reference Figure 1 The intelligent query method for industrial data based on a large language model includes steps S1-S6, as detailed below:

[0027] S1: Use a large language model to extract the semantic vector sequence of the target user's current query text and the text of each dimension of each data table in the database, and calculate the relevance vector between the target user's current query text and each data table based on the semantic vector sequence.

[0028] Taking any user as the target user, obtain the current query text input by the target user, and extract the metadata of each data table in the database, including: table name, table comments, field names, and field comments. Taking any data table in the database as the target data table, perform word segmentation on the current query text and the text of the target data table in the aforementioned four dimensions, respectively, to obtain the word sequence corresponding to the current query text and the text of each dimension.

[0029] The semantic vector of each word in the word sequence is extracted using a pre-trained large language model. Large language models can be such as Word2Vec (Word to Vector) or BERT (Bidirectional Encoder Representations from Transformers) to obtain the semantic vector sequence corresponding to the current query text and the text of each dimension in the target data table.

[0030] The relevance vector between the current query text and the target data table is calculated based on the semantic vector sequence. The specific calculation method includes: taking any dimension of the target data table as the target dimension, calculating the cosine similarity value between the semantic vector of each word in the current query text and the semantic vector of each word in the target dimension text, and normalizing the cosine similarity value using the maximum value normalization method. The average of all normalized cosine similarity values ​​is taken as the relevance value between the current query text and the target dimension text. The relevance values ​​corresponding to the text in all dimensions form the relevance vector between the current query text and the target data table.

[0031] Similarly, the relevance vector between the current query text and all data tables can be obtained.

[0032] S2: Obtain the relevance vector between the target user's historical query text and each data table. For each historical query, calculate the contribution of each dimension of the data table in the historical query based on the relevance vector between the historical query text and each data table.

[0033] Obtain the target user's historical query text. Using the same method as in step S1, the relevance vector between the target user's historical query text and each data table can be obtained. The relevance vector between a single historical query text and a single data table contains the relevance value between the historical query text and the text of each dimension of the data table.

[0034] Using any historical query as the target query, obtain the first set of data tables selected by the target user and the second set of data tables not selected from the target query results; calculate the first average of the relevance values ​​between the target dimension text of each data table in the first set and the historical query text of the target query; calculate the second average of the relevance values ​​between the target dimension text of each data table in the second set and the historical query text of the target query; calculate the difference between the first average and the second average, and normalize this difference to obtain the contribution of the target dimension. The specific formula is as follows:

[0035]

[0036] In the formula, Indicates the first Dimensions of the data table in the next historical query The degree of contribution; This represents the normalization function, which makes the sum of the contributions of all dimensions equal to 1; This indicates calculating the average value; Indicates the first The first set of data tables selected by the target user in the historical query; This indicates that all data tables in the data table collection 1 will be traversed. Indicates the first Data table dimensions in the next historical query The relevance value between the text and the historical query text; This refers to the dimensions of each data table in Data Table Collection 1. Calculate the average relevance values ​​between the text and the historical query text; Indicates the first Iterate through all data tables that were not selected by the target user in the previous historical queries; This refers to the dimensions of each data table that do not belong to the data table set 1. The relevance values ​​between the target text and historical query texts are averaged. The tables not selected by the target user are all other tables in the database that were not selected.

[0037] The larger the value, the more likely the target user is to select a data table in terms of dimensions. The selected data table is clearly more relevant than the unselected data table, and the target user is likely to rely on the dimension. Make a choice; The smaller the value, the more similar the selected and unselected data tables are in terms of dimension. The correlation is similar across dimensions, and target users do not rely on this dimension when making choices. A negative number means that the target users tend to choose data tables that are less relevant in this dimension.

[0038] Similarly, we can determine the contribution of each dimension in the data table for each historical query. When the contribution of a certain dimension's text is significantly higher than that of other dimensions, it indicates that the target user highly relied on that dimension when making their selection in that historical query.

[0039] S3: Calculate the preference value for each dimension based on the correlation between the historical query text and the current query text, as well as the contribution of each dimension of the data table in each historical query.

[0040] The Dynamic Time Warping (DTW) algorithm is used to calculate the shortest path distance between the semantic vector sequences of historical query texts and the semantic vector sequence of the current query text. The shortest path distance is then negatively correlated and normalized to obtain the relevance. This relevance is multiplied by the contribution of the target dimension in the target query to obtain the first product corresponding to the target dimension. Finally, the first products corresponding to the target dimensions in all historical queries are summed to obtain the preference value for each target dimension. The specific formula is as follows:

[0041]

[0042] In the formula, Represents the dimensions in the data table Preference values; This represents the total number of all historical queries. Indicates the first Dimensions of the data table in the next historical query The degree of contribution; A sequence of semantic vectors representing the current query text; Indicates the first Semantic vector sequence of the historical query text; This indicates that the DTW algorithm is used to calculate the sum of the current query text and the first query. The shortest path distance between the semantic vector sequences of the previous historical query texts. The smaller the distance, the more similar the two query texts are; the larger the distance, the less related they are. This represents the negative correlation normalization function, which aims to make the normalized output value larger as the distance between two query texts is smaller, and the sum of the normalized output values ​​between the semantic vector sequences of all historical query texts and the current query text is 1.

[0043] Based on the shortest path distance, or similarity, between the semantic vector sequences of historical query text and the current query text, the dimensions of a single historical query are analyzed. The contribution of historical query texts that are highly similar to the current query text is weighted. The contribution of historical query texts that are not similar to the current query text is given high weight; their dimensions are assigned lower weight. The contribution of [the element] is given low weight or even ignored. The final dimension obtained is [the following]. The preference value takes into account the preference information of all historical queries of the target user, and places more emphasis on historical query text that is more similar to the current query text.

[0044] S4: For each data table, calculate the basic score of the data table based on the relevance value of the text of each dimension to the current query text and the preference value of each dimension.

[0045] For a single data table, the preference values ​​for all dimensions are normalized so that the sum of all normalized preference values ​​is 1. For each dimension, the relevance value between the text of that dimension and the current query text is multiplied by the normalized preference value for that dimension to obtain the second product for that dimension. The base score of the data table is obtained by summing the second products for all dimensions. The specific formula is as follows:

[0046]

[0047] In the formula, Indicates the first The base score of each data table; This indicates the total number of dimensions in the data table. In this embodiment, ; Represents the dimensions in the data table Preference values; Indicates the first Dimensions of each data table The relevance value between the text and the current query text; This represents the normalization function, used to make the sum of the normalized preference values ​​for all dimensions equal to 1.

[0048] Since the target user's preference values ​​for each dimension are derived from their historical query behavior, different users will have significantly different preferences for each dimension. When dealing with the same current query text, the system assigns differentiated weights to the relevance values ​​of each dimension in the data table based on the different users' preferences for each dimension, so that the same data table receives different base scores in different user scenarios. Compared to the undifferentiated mode of existing technologies that return the same data table ranking results for the same query text, this solution achieves personalized recommendations tailored to each user, with recommendation results deeply adapted to individual user query habits.

[0049] S5: Calculate the inter-table similarity between this data table and each other data table. Use the base score of each other data table as a weight to perform a weighted fusion of the inter-table similarity. Combine the frequency with which this data table is selected by the target user to calculate the relative importance factor of this data table.

[0050] For each data table, the number of times it is selected by all users is obtained and normalized to obtain the normalized count. The relevance vector of this data table to the current query text and the normalized count of this data table together constitute the influencing factor sequence. The cosine similarity value between the influencing factor sequences of this data table and each other data table is calculated to obtain the inter-table similarity. The basic score of each other data table is used as a weight to perform a weighted fusion of the corresponding inter-table similarities to obtain a weighted fusion value. The probability that this data table is selected by the target user is calculated, and this probability is multiplied by the weighted fusion value to obtain a third product. The third product is normalized to obtain the relative importance factor. The specific formula is as follows:

[0051]

[0052] In the formula, Indicates the first The relative importance factors of each data table; Indicates the first The sequence of influencing factors in a data table , Indicates the first The relevance scores of the text in each of the four dimensions—table name, table comment, field name, and field comment—to the current query text are calculated as follows: , Indicates the first The probability that a data table is selected by all users Indicates the first The number of times each data table is selected by all users This represents the global maximum number of times each data table is selected by each user. Indicates the first The sequence of influencing factors, structural composition, and... same; express and The cosine similarity value between the corresponding influencing factor sequences, i.e., the inter-table similarity; Indicates the first The base score of each data table; Indicates the first The number of times each data table is selected by the target user; This represents the maximum number of times each data table was selected by the target user. This represents the normalization function, used to map the relative importance factors of all data tables to a uniform scale.

[0053] Reflects the first The relative importance of a data table in the target user's historical queries is assessed by a ratio; a higher ratio indicates a stronger reliance on that data table by the target user. By incorporating users' long-term query behavior habits into the data table importance evaluation process, different users will obtain differentiated relative importance factors for the same current query text due to their varying historical selection preferences. This, in turn, influences the final ranking results, enabling the recommendation system to adaptively adapt to individual user query habits and significantly improve the level of personalized service.

[0054] By using the base score The weighted fusion mechanism based on inter-table similarity allows the importance of high-score data tables to be transmitted along the inter-table similarity relationship to other related data tables. Even if a data table itself has a low semantic matching degree with the current query text (i.e., a low basic score), as long as it has a high semantic similarity with several high-score data tables, it can still obtain a high relative importance factor and thus be included in the recommendation candidate set. This effectively avoids the fragmentation problem of recommendation results caused by semantic matching theory and improves the overall exposure rate of related data tables.

[0055] S6: Multiply the base score of the data table by the relative importance factor to obtain the final score of the data table, and sort all data tables in descending order of the final score for target users to query.

[0056] The final score of the data table is obtained by multiplying its base score by the relative importance factor. The specific formula is as follows:

[0057]

[0058] In the formula, Indicates the first The final score of each data table; Indicates the first The base score of each data table; Indicates the first The relative importance factors of each data table.

[0059] After obtaining the final scores for all data tables, sort all data tables in descending order of the final scores for the target user to query.

[0060] By multiplying the base score by the relative importance factor, a final score of zero will result in zero if either the base score or the relative importance factor is zero. In other words, if a data table is completely semantically irrelevant to the current query text... ), or has no association with any high-basic score table and has never been used by the user ( If a data table performs poorly in terms of semantic matching, inter-table relationships, and user preferences, the multiplicative fusion algorithm significantly boosts its final score, giving it a dominant position in the ranking and ensuring that the highest quality, most relevant, and most user-friendly data tables are presented first. This invention significantly improves data table retrieval efficiency and user experience.

[0061] This application also discloses an intelligent industrial data query system based on a large language model. The system includes a processor and a memory. The memory stores computer program instructions. When the computer program instructions are executed by the processor, the intelligent industrial data query method based on a large language model according to the above embodiments of the present invention is implemented.

[0062] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0063] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An intelligent query method for industrial data based on a large language model, characterized in that, include: A large language model is used to extract semantic vector sequences of the target user's current query text and the text of each dimension of each data table in the database, and the relevance vector between the target user's current query text and each data table is calculated based on the semantic vector sequences. Obtain the relevance vector between the target user's historical query text and each data table. For each historical query, calculate the contribution of each dimension of the data table in the historical query based on the relevance vector between the historical query text and each data table. Calculate the correlation between the historical query text and the current query text. Calculate the preference value of each dimension based on the correlation between the historical query text and the current query text and the contribution of each dimension of the data table in each historical query. For each data table, a base score for that data table is calculated based on the relevance vector between that data table and the current query text, as well as the preference value for each dimension. Calculate the inter-table similarity between this data table and each other data table, use the base score of each other data table as a weight to perform a weighted fusion of the inter-table similarity, and combine the frequency of this data table being selected by the target user to calculate the relative importance factor of this data table; Multiply the base score of the data table by the relative importance factor to obtain the final score of the data table. Sort all data tables in descending order of the final score for target users to query.

2. The intelligent query method for industrial data based on a large language model according to claim 1, characterized in that, The calculation of the relevance vector between the target user's current query text and each data table based on the semantic vector sequence includes: taking any dimension of the data table as the target dimension, for each data table, calculating the cosine similarity value between each semantic vector in the semantic vector sequence of the current query text and each semantic vector in the semantic vector sequence of the target dimension text, and taking the average of all normalized cosine similarity values ​​as the relevance value between the current query text and the target dimension text; the relevance values ​​corresponding to the text of all dimensions form the relevance vector between the current query text and the data table.

3. The intelligent query method for industrial data based on a large language model according to claim 2, characterized in that, The method for calculating the contribution of each dimension of the data table in this historical query includes: obtaining the first set of data tables selected by the target user and the second set of data tables not selected in this historical query; obtaining the relevance value between the target dimension text of the data table and the text of this historical query from the relevance vector, and calculating the first average value of the relevance value between the target dimension text of each data table in the first set of data tables and the text of this historical query; calculating the second average value of the relevance value between the target dimension text of each data table in the second set of data tables and the text of this historical query; calculating the difference between the first average value and the second average value, and normalizing the difference to obtain the contribution of the target dimension.

4. The intelligent query method for industrial data based on a large language model according to claim 2, characterized in that, The calculation method for the preference value of each dimension includes: for each historical query, the DTW algorithm is used to calculate the shortest path distance between the semantic vector sequence of the historical query text and the semantic vector sequence of the current query text, and the shortest path distance is negatively correlated and normalized to obtain the relevance; the relevance is multiplied by the contribution of the target dimension in the historical query to obtain the first product corresponding to the target dimension in the historical query; the first products corresponding to the target dimensions in all historical queries are summed to obtain the preference value of the target dimension.

5. The intelligent query method for industrial data based on a large language model according to claim 2, characterized in that, The calculation method for the basic score of this data table includes: normalizing the preference values ​​of all dimensions of the data table so that the sum of all normalized preference values ​​is 1; for each dimension, multiplying the relevance value of the text of that dimension to the current query text by the normalized preference value of that dimension to obtain the second product corresponding to that dimension; summing the second products corresponding to all dimensions to obtain the basic score of the data table.

6. The intelligent query method for industrial data based on a large language model according to claim 1, characterized in that, The method for calculating the relative importance factor of this data table includes: obtaining the number of times the data table is selected by all users and normalizing it to obtain the normalized number; the relevance vector of the data table and the current query text, together with the normalized number of the data table, constitute the influencing factor sequence; calculating the cosine similarity value between the influencing factor sequence of the data table and each other data table to obtain the inter-table similarity; using the basic score of each other data table as a weight to perform weighted fusion of the corresponding inter-table similarities to obtain the weighted fusion value; calculating the probability that the data table is selected by the target user, multiplying the probability by the weighted fusion value to obtain the third product, and normalizing the third product to obtain the relative importance factor.

7. The intelligent query method for industrial data based on a large language model according to claim 1, characterized in that, Extracting the semantic vector sequence of the target user's current query text using a large language model includes: segmenting the current query text into words to obtain a word sequence; and using a pre-trained large language model to extract the semantic vector of each word in the word sequence to obtain the semantic vector sequence of the current query text.

8. The intelligent query method for industrial data based on a large language model according to claim 1, characterized in that, Each data table has four dimensions of text: table name, table comment, field name, and field comment.

9. An intelligent industrial data query system based on a large language model, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement the intelligent industrial data query method based on a large language model according to any one of claims 1-8.

Images