A query statement semantic tree generation encoding method and system

Abstract

The present invention belongs to the field of natural language processing, and provides a query statement semantic tree generation encoding method and system, obtains a given query statement, and classifies it according to the completeness of the query statement; for the incomplete query statement, uses the process completion large language model to complete the query process based on the guidance of the thinking chain; based on the complete query statement or the completed query statement, extracts the deep semantic association between adjacent words through the component analysis tree, extracts the long-distance association of non-adjacent words through the dependency syntax tree, and fuses the dependency syntax tree and the component analysis tree to form a final semantic tree. The present invention classifies the query statement by completeness, and then uses the large language model to automatically infer and improve the query statement, thereby generating a semantic tree that can accurately reflect the intermediate process of the action, realizes the supplement of key semantic content, and can support accurate action retrieval and positioning in open scene videos.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and specifically relates to a query statement semantic tree generation encoding method and system, and in particular to a query statement semantic tree generation encoding method and system based on adaptive process completion. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] As video data continues to grow, accurate positioning of video content is the key to improving video viewing efficiency. In terms of accurate positioning of video content, there are many problems that need to be solved.

[0004] In an open environment, the length of query statements and the level of semantic information vary widely. In particular, for incomplete query statements, existing methods are usually unable to fully supplement the detailed information of the action content in the query statement when constructing the semantic tree. When performing query representation for video clip positioning tasks, semantic trees are usually constructed through simple similarity calculations or grammatical dependency tree parsing tools. Due to incomplete queries, lack of a large amount of process information, lack of supplementation of missing process information in the grammatical tree construction process, and lack of in-depth exploration of the long- and short-range grammatical relationships between words and phrases, this will lead to serious query semantic understanding deviations in open scenarios. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a query statement semantic tree generation encoding method and system. The present invention classifies the query statement by completeness, and then uses a large language model to automatically infer and improve the query statement, thereby generating a semantic tree that can accurately reflect the intermediate process of the action, supplementing the key semantic content, and supporting accurate action retrieval and positioning in open scene videos.

[0006] According to some embodiments, the present invention adopts the following technical solutions:

[0007] A query statement semantic tree generation encoding method comprises the following steps:

[0008] Get the given query statement and classify it according to the completeness of the query statement;

[0009] For incomplete query statements, the process completion language model is used to complete the query process based on the guidance of thought chain.

[0010] Based on the complete query statement or the completed query statement, the deep semantic association between adjacent words is extracted through the component analysis tree, the long-distance association between non-adjacent words is extracted through the dependency syntax tree, and the dependency syntax tree and the component analysis tree are fused to form the final semantic tree.

[0011] As an optional implementation, the process of classifying the query statement according to its completeness includes: using a classifier to classify the given query statement and assigning classification labels, wherein the first type of label indicates that the query statement is complete and a semantic tree can be directly constructed; the second type of label indicates that the query statement has medium completeness and requires at least a single-step completion process; the third type of label indicates that the query statement is incomplete and requires a multi-step completion process.

[0012] As an optional implementation, the classifier is a trained small language model, and is trained using cross entropy loss during the training process.

[0013] As an optional implementation, the process of completing the query process based on the guidance of the thought chain by using the process completion large language model includes:

[0014] Based on the query statement, the large language model is prompted to generate step-by-step thinking under zero-shot conditions, and each thinking step generated is modified using retrieval-augmented generation until the final answer is generated.

[0015] Furthermore, the process of using retrieval enhancement to generate and modify each thinking step includes: assuming that the previous thinking step has been fixed and the current thinking step needs to be revised , from the text Transformed into a query start:

[0016] ;

[0017] in, Representative Thinking steps, is a large language model that transforms the given original query , current and past thought steps ,… Converted into query statements that can be processed by the retrieval system ; Use retrieval enhancement to generate query statements Retrieve related documents , add it to the query, and the revised thinking steps are:

[0018] ;

[0019] in Given the original query Q, think about the steps , and related documents Under the condition of The probability distribution of .

[0020] The large language model is prompted to generate a complete response from the revised thinking steps in a step-by-step manner.

[0021] As a further step, the process of adopting retrieval-enhanced generation includes:

[0022] Extract semantic-aware embeddings of given candidate documents and query statements, and calculate the relevance between the query statement and candidate documents;

[0023] Select multiple candidate documents with the highest relevance.

[0024] As an optional implementation, the process of extracting the deep semantic association between adjacent words through the component analysis tree and extracting the long-distance association of non-adjacent words through the dependency syntax tree includes:

[0025] Extract the word vector, part-of-speech tag, and depth of each word in the component analysis tree to form an array;

[0026] Constructing an attention mechanism based on a component analysis tree to capture the deep semantic association between adjacent words, the attention mechanism includes the semantic similarity between the word vectors of the associated words, the part-of-speech combination weights, and the grammatical association coefficient generated by the depth difference of the associated words in the component analysis tree;

[0027] The two ends of the relation entity based on the dependency syntactic tree are defined as the long-range dependency association between words, and the long-range semantic association coefficient between words based on the dependency syntactic tree is calculated;

[0028] According to the grammatical association coefficient and the long-range semantic association coefficient, the component analysis tree and the dependency syntax tree are updated.

[0029] A query statement semantic tree generation encoding system, comprising:

[0030] The classification module is configured to obtain a given query statement and classify it according to the completeness of the query statement;

[0031] The completion module is configured to complete the incomplete query statement using the process completion large language model based on the thought chain guidance query process;

[0032] The semantic tree generation module is configured to extract the deep semantic associations between adjacent words through the component analysis tree based on the complete query statement or the completed query statement, extract the long-distance associations between non-adjacent words through the dependency syntax tree, and fuse the dependency syntax tree and the component analysis tree to form the final semantic tree.

[0033] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the steps in the above method are completed.

[0034] An electronic device comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor. When the computer instructions are executed by the processor, the steps in the above method are completed.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention uses a large language model and an external knowledge base to perform appropriate semantic completion for incomplete or simple queries; for completed queries, a semantic tree is constructed based on long- and short-range relationship mining, and long-range semantic associations based on relationship entities and semantic associations between adjacent words based on component analysis trees are established to strengthen the semantic influence of directly related phrases and weaken the semantic contribution of non-directly related phrases, thereby ensuring the accuracy and precision of the semantic tree generation results.

[0037] The present invention completes the given query statement based on an adaptive process to supplement the process information missing in the query, thereby ensuring a more accurate result. It can solve the problem of semantic deviation caused by incomplete queries and other reasons when performing query representation for video clip positioning tasks in open scenarios.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0040] Figure 1 The present invention is a schematic diagram of a query statement semantic tree generation encoding process according to an embodiment. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0045] Embodiment 1

[0046] A query statement semantic tree generation encoding method based on adaptive process completion, such as Figure 1 As shown, the following steps are included:

[0047] Step S1. Given a query statement (hereinafter referred to as query) Q, the query statement Q is divided into three categories: A, B, and C by using a classifier, and different strategies are used to process queries of different completeness;

[0048] Step S2. For incomplete queries, query process completion based on thought chain guidance is performed. The process completion large language model can generate step-by-step ideas in the zero-sample reasoning process. Complete the missing process information in the query.

[0049] Step S3. Obtaining the complete query Afterwards, a semantic tree was constructed based on long- and short-range relationship mining. By utilizing the different features and relationships of the syntactic tree and the distribution characteristics of word vectors, a complex semantic association network was established to fully understand and mine the semantic information in the text.

[0050] The specific steps are as follows:

[0051] Step S1 specifically includes the following steps:

[0052] S11. Determine the completeness of the query statement:

[0053] Build a classifier that returns the appropriate level of completeness (or completeness, or complexity) for a given query.

[0054] Specifically, given the query , the classifier can be expressed as follows: ,in, is a small language model trained to classify three different complexity levels, is its corresponding category label.

[0055] There are three class labels: 'A', 'B', and 'C', where 'A' means It is complete and can directly construct the semantic tree; 'B' means With medium completeness, at least a single step of step S2 is required to complete the semantics; 'C' means It is incomplete and requires a multi-step semantic completion process of step S2.

[0056] S12. Small language model (i.e. classifier) ​​training:

[0057] In order to accurately determine the completeness of the query and determine the required labels, multiple classifiers need to be trained. In this embodiment, the classifier is trained using cross entropy loss, and the completeness of the query statement is determined by the classifier during reasoning, that is, one of {"A", "B", "C"}: .

[0058] The specific process of step S2 includes:

[0059] First, let's introduce how to obtain relevant document information through retrieval:

[0060] Given a set of candidate documents , where I represents the number of documents. The goal of retrieval-enhanced generation is to retrieve the most relevant candidate documents based on a query statement Q, which can be the question and task prompt itself, or related information generated by a large language model. To achieve this, the retrieval-enhanced generation method first extracts the documents , and query statements The semantic-aware embedding of This can be achieved through various text embedding models. is the embedding vector of the document, q is the embedding vector of the query, and K is the dimension of the vector space. The relevance between the query and the document is measured by their cosine similarity, that is:

[0061] ;

[0062] The top ranked ones are based on their relevance. documents are input into the prompt so that the large language model can generate the final answer.

[0063] Here is the process of generating a correct answer response:

[0064] Use different documents to generate each candidate word. For the i-th position, the probability of the candidate word is equal to the sum of the conditional probabilities of all documents, that is, calculate the marginal probability of the candidate word for the document, that is:

[0065] ;

[0066] in, are the k documents with the highest relevance ranking, is the probability that R document is retrieved given the query statement Q, N is the total length of the reply sequence, Given the query statement Q, document R and the first i-1 candidate words, generate the candidate word w i probability.

[0067] The following are the specific steps of the query process completion process based on the guidance of the thinking chain:

[0068] Given a query sentence Q, prompt the large language model to generate step-by-step thinking under zero-shot conditions, ,in Representative A thinking step.

[0069] because may be flawed (e.g., contain hallucinations), in this embodiment, each thought step generated is revised using retrieval enhancement generation, and then the final answer is generated from these thoughts.

[0070] Specifically, suppose that the previous thinking steps have been fixed and are now about to be revised , you can get the document Transformed into a query start:

[0071]

[0072] in is a large language model that converts the original query , current and past thought steps ,… Converted into query statements that can be processed by the retrieval system . Using retrieval-enhanced generation, Retrieve related documents , and then add it to the prompt to generate the revised thinking steps : ;

[0073] in Given the original query Q, think about the steps , and related documents Under the condition of The probability distribution of .

[0074] Finally, according to the actual task, the revised complete thinking steps including m steps can simply be used as the final model response, and the large language model will be further prompted to generate a complete response from each revised thought step by step in a step-by-step manner.

[0075] It should be noted that in the revision Thinking Steps When using not only the current step or a complete chain of thought to generate queries for retrieval enhancement, but to ensure that the query From the current thinking step and the previously revised thinking steps generated in the process, that is, using the method of causal reasoning to correct these thinking steps through retrieval enhancement generation:

[0076] ;

[0077] ;

[0078] Revise original thinking by constantly referring to different textual materials errors in reasoning and ensures that each reasoning step is based on the most accurate and relevant information, significantly improving the quality and reliability of the generated output.

[0079] The process of step S3 includes:

[0080] S31. Extract word attributes:

[0081] For a completed input query This example uses two different semantic trees and dependency syntax trees. And component analysis tree Parse the sentence. Extract the word vector, part-of-speech tag, and The depth in forms a quad .

[0082] Among them, array W records the initial position of each word in the sentence to preserve the original sequence of words:

[0083] ;

[0084] Arrays Store the part-of-speech tags corresponding to all words in order:

[0085] ;

[0086] Array D records the depth of each word in the component analysis tree according to its position in the sentence:

[0087] ;

[0088] in, Indicates the first word, The total number of words in the sentence, express Part of speech, express The depth. Array express Middle Words relationship entity.

[0089] In this embodiment, only whether an entity relationship is formed between words is considered. If such a relationship is formed between words, the system will store these associated words and associated words, as well as the entity relationship name between them, in the form of triples P for storage.

[0090] S32. Short-range semantic association of adjacent words based on component analysis tree:

[0091] By building an attention mechanism based on a component analysis tree, this example strengthens the semantic connection between adjacent words in a sentence. This attention mechanism involves analyzing the semantic similarity between words, the weights of part-of-speech combinations, and the position differences in the component analysis tree, which together define the grammatical connection between words. Specifically, the semantic similarity between adjacent words is defined as:

[0092] ;

[0093] in, The function is the Cos similarity measure. In order to express the different contributions of two adjacent words to the grammatical structure of the text, different weights are set for different part-of-speech combinations. Indicates Word and The word part-of-speech combination weight is used to reflect the overall information of the associated word in the corpus, where In the component analysis tree, the grammatical correlation coefficient generated by the difference in the depth of the associated words is expressed as Indicates; using tree depth difference Indicates The depth of words and The absolute value of the difference in word depths.

[0094] ;

[0095] In order to distinguish the different semantic and grammatical associations between words, this example uses the grammatical association coefficient Different values ​​are used to represent the depth difference.

[0096] ;

[0097] in, , , y are adjustable constants. The correlation coefficient between adjacent words based on the component analysis tree is defined as:

[0098] .

[0099] S33. Long-range semantic associations between words based on dependency syntactic trees:

[0100] When the sentence structure is more complex, the modification relationship may exist between non-adjacent words. In order to resolve such long-distance word relationships, this example uses the dependency syntax tree to define the dependency relationships between words and stores these relationships in the array P. and The long-range dependencies of express.

[0101] ;

[0102] ;

[0103] in, Expressing phrases and Whether there is a relationship entity between them is a binary bit. and If there is a relationship entity between ;otherwise .

[0104] The long-range semantic correlation coefficient between words based on the dependency syntactic tree is defined as:

[0105] ;

[0106] in, . Through the correlation coefficient between adjacent words and the long-range semantic association coefficient , update the component analysis tree and dependency syntax trees ,get and This embodiment directly integrates the dependency syntax tree And component analysis tree Forming the final semantic tree .

[0107] For the semantic tree, this embodiment is based on the complete query The word-level representation in is used to update the word features based on the semantic tree. ,in is the node set containing all words, is the set of edges containing directed syntactic dependencies between words in all queries. Formally, there are layer The graph network structures are stacked together, where The layer graph structure can be expressed as:

[0108] ;

[0109] Among them, ReLU is a nonlinear activation function; for the Layer graph structure, It is The input features of the node words, is the learned node feature and The edge weights between is the trainable weight matrix of this layer, Representing all and Neighbor nodes with semantic relationship between node words; It is The output of the layer graph network is then input to the The layers are structured the same as in the diagram.

[0110] Representing the query at word level After being initialized as the input of the first graph, it can finally be obtained by The layer graph obtains a syntax-aware word-level query representation, namely .

[0111] Embodiment 2

[0112] A query statement semantic tree generation encoding system, comprising:

[0113] The classification module is configured to obtain a given query statement and classify it according to the completeness of the query statement;

[0114] The completion module is configured to complete the incomplete query statement using the process completion large language model based on the thought chain guidance query process;

[0115] The semantic tree generation module is configured to extract the deep semantic associations between adjacent words through the component analysis tree based on the complete query statement or the completed query statement, extract the long-distance associations between non-adjacent words through the dependency syntax tree, and fuse the dependency syntax tree and the component analysis tree to form the final semantic tree.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.

Claims

1. A query statement semantic tree generation encoding method, characterized in that: The following steps are involved: Get the given query statement and classify it according to the completeness of the query statement; For incomplete query statements, the process completion language model is used to complete the query process based on the guidance of thought chain. The process of using the process completion large language model to complete the query process based on the guidance of the thought chain includes: Based on the query, the large language model is prompted to generate step-by-step thinking under zero-shot conditions, and each step of the generated thinking is modified using retrieval-enhanced generation until the final answer is generated. The process of using retrieval enhancement to generate and modify each thinking step generated includes: assuming that the previous thinking step is fixed, the current thinking step needs to be revised , from the text Transformed into a query start: ； in, Representative Thinking steps, is a large language model that transforms the given original query , current and past thought steps ,… Converted into query statements that can be processed by the retrieval system ; Use retrieval enhancement to generate query statements Retrieve related documents , add it to the query, and the revised thinking steps are: ； in Given the original query Q, think about the steps , and related documents Under the condition of The probability distribution of The large language model is prompted to gradually generate a complete response from each revised thinking step; Based on the complete query statement or the completed query statement, the deep semantic association between adjacent words is extracted through the component analysis tree, and the long-distance association between non-adjacent words is extracted through the dependency syntax tree. The dependency syntax tree and the component analysis tree are fused to form the final semantic tree.

2. A query statement semantic tree generation encoding method as claimed in claim 1, characterized in that: According to the completeness of the query statement, the process of classifying it includes: using a classifier to classify the given query statement and assigning classification labels, wherein the first type of label indicates that the query statement is complete and can directly construct a semantic tree; the second type of label indicates that the query statement has medium completeness and requires at least a single-step completion process; the third type of label indicates that the query statement is incomplete and requires a multi-step completion process.

3. A query statement semantic tree generation encoding method as claimed in claim 2, characterized in that: The classifier is a trained small language model and is trained using cross entropy loss during training.

4. A query statement semantic tree generation encoding method as claimed in claim 1, characterized in that: The process of using retrieval-augmented generation includes: Extract semantic-aware embeddings of given candidate documents and query statements, and calculate the relevance between the query statement and candidate documents; Select multiple candidate documents with the highest relevance.

5. The query statement semantic tree generation encoding method as claimed in claim 1, characterized in that: The process of extracting the deep semantic associations between adjacent words through the component analysis tree and the long-distance associations between non-adjacent words through the dependency syntax tree includes: Extract the word vector, part-of-speech tag, and depth of each word in the component analysis tree to form an array; Constructing an attention mechanism based on a component analysis tree to capture the deep semantic association between adjacent words, the attention mechanism includes the semantic similarity between the word vectors of the associated words, the part-of-speech combination weights, and the grammatical association coefficient generated by the depth difference of the associated words in the component analysis tree; The two ends of the relation entity based on the dependency syntactic tree are defined as the long-range dependency association between words, and the long-range semantic association coefficient between words based on the dependency syntactic tree is calculated; According to the grammatical association coefficient and the long-range semantic association coefficient, the component analysis tree and the dependency syntax tree are updated.

6. A query statement semantic tree generation encoding system, characterized in that: include: The classification module is configured to obtain a given query statement and classify it according to the completeness of the query statement; The completion module is configured to complete the incomplete query statement using the process completion large language model based on the thought chain guidance query process; The process of using the process completion large language model to complete the query process based on the guidance of the thought chain includes: Based on the query, the large language model is prompted to generate step-by-step thinking under zero-shot conditions, and each step of the generated thinking is modified using retrieval-enhanced generation until the final answer is generated. The process of using retrieval enhancement to generate and modify each thinking step generated includes: assuming that the previous thinking step is fixed, the current thinking step needs to be revised , from the text Transformed into a query start: ； in, Representative Thinking steps, is a large language model that transforms the given original query , current and past thought steps ,… Converted into query statements that can be processed by the retrieval system ; Use retrieval enhancement to generate query statements Retrieve related documents , add it to the query, and the revised thinking steps are: ； in Given the original query Q, think about the steps , and related documents Under the condition of The probability distribution of The large language model is prompted to gradually generate a complete response from each revised thinking step; The semantic tree generation module is configured to extract the deep semantic associations between adjacent words through the component analysis tree based on the complete query statement or the completed query statement, extract the long-distance associations between non-adjacent words through the dependency syntax tree, and fuse the dependency syntax tree and the component analysis tree to form the final semantic tree.

7. A computer-readable storage medium, characterized in that: Used to store computer instructions, which, when executed by a processor, complete the steps of the method according to any one of claims 1 to 5.

8. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the steps in the method according to any one of claims 1 to 5 are completed.

Images