Query analysis method and system, computer readable storage medium and terminal

Abstract

The present invention provides a query analysis method and system, a computer-readable storage medium and a terminal, and relates to the field of machine learning. The method includes: determining a vector corresponding to a sentence to be queried, and obtaining a vector to be queried; inputting the vector to be queried into a trained first model, and obtaining a pseudo title vector according to the output of the first model, wherein the pseudo title vector contains the posterior retrieval information of the sentence to be queried; inputting the sentence to be queried and the pseudo title vector into a trained second model, and determining the target prediction of the sentence to be queried according to the output of the second model, and performing query analysis based on the target prediction. This technical solution takes the query classification method in query analysis as an example, introduces the posterior retrieval information into the prior retrieval information scenario, so that the classifier performance is further improved compared with the previous query classification method, thereby enhancing the performance of query analysis.

Description

Technical Field

[0001] The present disclosure relates to the field of machine learning, and in particular to a query analysis method and system, a computer-readable storage medium, and a terminal. Background Art

[0002] Query analysis methods are a key component of search engines, providing guidance for the recall and ranking stages. As query analysis discovers more semantic information, search engines can more accurately understand users' information needs and provide users with more satisfactory search results accordingly. In recent years, more and more research efforts have been devoted to improving the efficiency of query analysis tasks.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a query analysis method, which at least to some extent overcomes the problem that the efficiency of query analysis in the related art needs to be improved.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to a first aspect of the present disclosure, a query analysis method is provided, the method comprising: determining a vector corresponding to a sentence to be queried to obtain the vector to be queried; inputting the vector to be queried into a trained first model, and obtaining a pseudo title vector according to an output of the first model, wherein the pseudo title vector includes a posteriori retrieval information of the sentence to be queried; inputting the sentence to be queried and the pseudo title vector into a trained second model, and determining a target prediction of the sentence to be queried according to an output of the second model, and performing query analysis based on the target prediction.

[0007] In one embodiment of the present disclosure, the first model includes a first coding model and a second coding model, and the method includes: obtaining N training samples, the i-th training sample includes an i-th query statement and an i-th title set corresponding to the i-th query statement, N is a positive integer, and i is a positive integer not greater than N; inputting the i-th query statement into the first coding model to obtain an i-th query vector corresponding to the i-th query statement; inputting the i-th title set into the second coding model to obtain an i-th title vector corresponding to the i-th title set; determining a first loss function of the first model according to the i-th query vector and the i-th title vector; optimizing parameters of the first coding model and parameters of the second coding model according to the first loss function to determine the trained first model.

[0008] In one embodiment of the present disclosure, the first loss function calculation method is: Among them, minimize Loss imi Indicates the calculation of the first loss function Loss imi Minimize q i is the i-th query vector mentioned above, t i is the i-th title vector mentioned above.

[0009] In one embodiment of the present disclosure, the above-mentioned inputting the above-mentioned query vector into the trained first model and obtaining the pseudo title vector according to the output of the above-mentioned first model includes: inputting the above-mentioned query vector into the first encoding model in the above-mentioned trained first model, and obtaining the above-mentioned pseudo title vector according to the output of the above-mentioned first encoding model.

[0010] In one embodiment of the present disclosure, the first model includes a third encoding model and a sequence-sequence model, and the method further includes: obtaining N training samples, the i-th training sample includes an i-th query statement and an i-th title set corresponding to the i-th query statement, N is a positive integer, and i is a positive integer not greater than N; inputting the i-th query statement into the third encoding model to obtain a context vector corresponding to the i-th query statement; obtaining a group of first hidden states corresponding to the context vector through the encoder in the sequence-sequence model; using the last hidden state of the first hidden state as the i-th pseudo title vector corresponding to the i-th query statement; determining a group of first probability vectors corresponding to the i-th title set; inputting the first hidden state and the first probability vector into the decoder in the sequence-sequence model to obtain a group of second hidden states according to the output of the decoder; inputting the second hidden state into a normalized exponential function to obtain a group of second probability vectors according to the output of the normalized exponential function; determining a second loss function of the first model according to the cross entropy function of the first probability vector and the second probability vector; optimizing the parameters of the third encoding model and the parameters of the sequence-sequence model according to the second loss function to determine the trained first model.

[0011] In one embodiment of the present disclosure, the second loss function calculation method is: Among them, minimizeLoss′ imi Indicates the calculation of the second loss function Loss′ imi Minimization of is a set of first probability vectors corresponding to the above i-th title set, is a set of second probability vectors corresponding to the above i-th title set, CE(·) is the above cross entropy function, O is the length of all the above title sets, O is a positive integer, and l is a positive integer not greater than O.

[0012] In one embodiment of the present disclosure, the above-mentioned input of the query vector into the trained first model and obtaining the pseudo title vector according to the output of the above-mentioned first model include: inputting the above-mentioned query vector into the third encoding model in the above-mentioned trained first model to obtain the context vector corresponding to the above-mentioned query vector; obtaining a group of first hidden states corresponding to the above-mentioned context vector through the encoder in the sequence-sequence model in the above-mentioned trained first model; and determining the last hidden state in the above-mentioned group of first hidden states as the above-mentioned pseudo title vector.

[0013] In one embodiment of the present disclosure, the method further includes: obtaining the i-th label vector corresponding to the i-th query statement to obtain N training samples; obtaining the i-th pseudo title vector corresponding to the i-th query statement obtained in the first model; embedding the i-th query statement to obtain the i-th embedding sequence; combining the i-th pseudo title vector and the i-th embedding sequence, and inputting the combined vector into a fourth encoding model; obtaining the i-th target representation of the i-th query statement according to the output of the fourth encoding model; inputting the i-th target representation into the normalized exponential function, and obtaining the target prediction according to the output of the normalized exponential function; determining the third loss function of the second model according to the cross entropy function of the i-th label vector and the target prediction; optimizing the parameters of the fourth encoding model according to the third loss function to determine the trained second model.

[0014] In one embodiment of the present disclosure, the third loss function calculation method is: Among them, minimizeLoss int Indicates the calculation of the third loss function Loss int Minimization of To predict the above goals, is the i-th label vector mentioned above, there are G label vectors mentioned above, G is a positive integer, and j is a positive integer not greater than G.

[0015] In one embodiment of the present disclosure, the method further includes: determining a target loss function based on the third loss function and the first loss function or the second loss function; and optimizing the first model and the second model based on the target loss function.

[0016] In one embodiment of the present disclosure, the target loss function calculation method is: minimizeLoss=Loss int +αLoss imi , or minimizeLoss = Loss int +αLoss′ imi, where minimizeLoss means calculating the above target loss function, Loss imi is the first loss function, Loss′ imi The second loss function mentioned above, Loss int is the third loss function mentioned above, and α is a hyperparameter.

[0017] In one embodiment of the present disclosure, the sentence to be queried and the pseudo title vector are input into the trained second model, and the target prediction of the sentence to be queried is determined according to the output of the second model, including: embedding the context vector corresponding to the vector to be queried to obtain an embedding sequence; combining the pseudo title vector and the embedding sequence, and inputting the combined vector into a fourth encoding model; obtaining the target representation of the sentence to be queried according to the output of the fourth encoding model; inputting the target representation into the normalized exponential function, and obtaining the target prediction according to the output of the normalized exponential function.

[0018] According to a second aspect of the present disclosure, a query analysis system is provided, the system comprising: a conversion module, used to determine a vector corresponding to a sentence to be queried, and obtain the vector to be queried; a simulation module, used to input the vector to be queried into a first model, and obtain a pseudo title vector according to an output of the first model, wherein the pseudo title vector includes a posteriori retrieval information of the sentence to be queried; an integration module, used to input the sentence to be queried and the pseudo title vector into a second model, and determine a target prediction of the sentence to be queried according to the output of the second model, and perform query analysis based on the target prediction.

[0019] According to a third aspect of the present disclosure, a terminal is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the query analysis method of the first aspect when executing the computer program.

[0020] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the query analysis method of the first aspect or the second aspect is implemented.

[0021] The query analysis method and system, computer-readable storage medium, and terminal provided by the embodiments of the present disclosure have the following technical effects:

[0022] In the query analysis process provided by the embodiment of the present disclosure, after determining the vector corresponding to the sentence to be queried, the vector to be queried is obtained, and the vector to be queried is input into the trained first model, and a pseudo title vector is obtained according to the output of the first model. Among them, the pseudo title vector contains the posterior retrieval information of the sentence to be queried. The sentence to be queried and the pseudo title vector are input into the trained second model, the target prediction of the sentence to be queried is determined according to the output of the second model, and query analysis is performed based on the target prediction. Taking the query classification method in query analysis as an example, this technical solution introduces the posterior retrieval information into the prior retrieval information scenario, so that the classifier performance is further improved compared with the previous query classification method, thereby improving the efficiency of query analysis and enhancing the performance.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0025] Figure 1 A schematic diagram showing the search engine process;

[0026] Figure 2 A statistical diagram schematically showing query frequency;

[0027] Figure 3 The structural diagram of the overall framework is schematically shown;

[0028] Figure 4 A flowchart of a query analysis method provided by an embodiment of the present disclosure is schematically shown;

[0029] Figure 5 The flowchart of the training phase of the detection method provided by the embodiment of the present disclosure is schematically shown;

[0030] Figure 6 The structure diagram of the detection method provided by the embodiment of the present disclosure is schematically shown;

[0031] Figure 7 A flowchart of a query analysis method provided by another exemplary embodiment of the present disclosure is schematically shown;

[0032] Figure 8 The flowchart of the training phase of the generation method provided by the embodiment of the present disclosure is schematically shown;

[0033] Fig. 9 The structure diagram of the generation method provided by the embodiment of the present disclosure is schematically shown;

[0034] Fig.10 A flowchart of a query analysis method provided by another exemplary embodiment of the present disclosure is schematically shown;

[0035] Fig.11 The flowchart of the integrated module training phase provided by the embodiment of the present disclosure is schematically shown;

[0036] Fig.12 A flowchart schematically shows a method for determining a target representation of a query statement provided by an embodiment of the present disclosure;

[0037] Fig.13 The structure diagram of the query analysis system provided by an embodiment of the present disclosure is schematically shown;

[0038] Fig.14 A block diagram of a query analysis terminal provided by an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0040] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are only examples of systems and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0041] In the description of the present disclosure, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances. In addition, in the description of the present disclosure, unless otherwise specified, "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship.

[0042] refer to Figure 1As shown in the search engine flow diagram, in a well-designed search engine, there are usually multiple query analysis modules (Query Analysis Modules), including a priori query analysis (pre-retrieval QA) module and a posteriori query analysis (post-retrieval QA) module. Since query statements are usually short, if only the meaning of the query itself is analyzed, the accuracy will be limited to a certain extent. Therefore, the query analysis module in the search engine will use some external information related to the query statement to enrich its meaning. Among them, external information can be divided into two types: prior retrieval information and post-retrieval retrieval information. Figure 1 As shown in the figure, the prior query analysis module is usually the first component that the input query passes through, and it provides sufficient prior retrieval information for the recall and ranking stages, such as query categories, query key concepts, etc. The posterior analysis module generates posterior retrieval information after the recall and ranking stages, such as document titles, summaries, and user click data, etc. This posterior retrieval information is of great help to the accuracy of query analysis.

[0043] Although the historical query logs of the prior query analysis module can be used to obtain a posteriori retrieval information online, this is only useful for queries that have already appeared. That is, historical logs cannot be used when processing new queries that appear for the first time. Figure 2 After statistically analyzing the search frequencies of 10 million queries randomly selected from a search engine, we plotted a frequency statistical graph of query frequency and its corresponding number of queries. We found that the frequency distribution is long-tailed, that is, a large number of queries are long-tail data with very few samples, while only a small number of queries have sufficient sample sizes and rich intra-class diversity. However, the limited number of samples is not enough to represent the true distribution of this category. Therefore, it is difficult to obtain a posteriori retrieval information for new queries to improve the performance of the query analysis module.

[0044] Due to the a priori retrieval scenario and long-tail data problems, it is likely that good a priori retrieval information cannot be obtained in the inference stage. Therefore, it is necessary to integrate the a priori retrieval information into the a priori query analysis model. In view of the above technical background and related technical shortcomings, this technical solution focuses on solving the technical problem of how to introduce a priori retrieval information into the limited a priori query analysis model to further improve the performance of the query analysis method.

[0045] Since there is a problem in the related art that the a posteriori retrieval information cannot be obtained in the inference stage, this technical solution assumes that the a posteriori retrieval information has a strong correlation with the priori retrieval information, and most of the a posteriori retrieval information can be inferred from the priori retrieval information. Specifically, Figure 3 A schematic diagram of an overall framework structure provided according to an embodiment of the present disclosure is shown.

[0046] refer to Figure 3 This technical solution proposes a PostIF framework, which consists of two parts: the Imitation Module and the Integration Module. The Imitation Module is responsible for constructing the pseudo-posteriori retrieval representation, and the Integration Module is responsible for effectively integrating the priori retrieval representation and the pseudo-posteriori retrieval representation. Each module has its own training phase and inference phase. The input of the training phase includes query statements and their corresponding titles, while the input of the inference phase includes only query statements.

[0047] For the simulation module, this technical solution introduces two pseudo-posteriori retrieval representation learning strategies: detection method and generation method. The purpose of the detection method is to maximize the similarity between the title representation and the query representation to learn the conversion between them; the purpose of the generation method is to achieve the conversion between the title and the query by directly generating a pseudo-title representation from the query under the guidance of the title.

[0048] There are many query analysis tasks, including query classification (input query statements, output their classification), query completion (input part of the query, complete the entire query), query conversion (find a synonymous query statement to replace the input query), etc. In this technical solution, the query classification task in query analysis is taken as an example for explanation.

[0049] In this technical solution, the prior retrieval information uses the query statement as the prior retrieval information, and the a posteriori retrieval information uses the top 5 search document titles. Thus, a query statement set Q = {Q1, Q2, ..., Q N}, there are N query statements, Q i represents the i-th query statement. For each query statement Q i , each has a corresponding set of titles There are M title sentences in total. represents the jth title of the ith query statement. In addition, each query statement set Q also corresponds to a label set Y = {Y1, Y2, ..., Y N},Y iIndicates the category label to which the i-th query statement belongs. A query statement Q i Can only belong to one category Y i , cannot belong to multiple categories.

[0050] In the training phase, the query set Q, title set T, and tag set Y are all available. However, in the inference phase, only the query set Q is available, which is used to infer the category of each query.

[0051] The following will describe each step of the query analysis method in this example implementation in more detail with reference to the accompanying drawings and embodiments.

[0052] in, Figure 4 The flowchart of the query analysis method according to an exemplary embodiment of the present disclosure is schematically shown. Figure 4 , the method comprises the following steps:

[0053] S401, determining a vector corresponding to a query statement to obtain a query vector;

[0054] S402, inputting the query vector into the trained first model, and obtaining a pseudo title vector according to the output of the first model, wherein the pseudo title vector includes a posteriori retrieval information of the query sentence;

[0055] S403, inputting the query sentence and the pseudo title vector into the trained second model, determining the target prediction of the query sentence according to the output of the second model, and performing query analysis based on the target prediction.

[0056] exist Figure 4 In the query analysis process provided by the illustrated embodiment, the vector corresponding to the sentence to be queried is determined to obtain the vector to be queried; the vector to be queried is input into the trained first model, and a pseudo title vector is obtained according to the output of the first model, wherein the pseudo title vector contains the a posteriori retrieval information of the sentence to be queried; the sentence to be queried and the pseudo title vector are input into the trained second model, and the target prediction of the sentence to be queried is determined according to the output of the second model, and query analysis is performed based on the target prediction. This technical solution takes the query classification method in query analysis as an example, introduces the a posteriori retrieval information into the a priori retrieval information scenario, so that the classifier performance is further improved compared with the previous query classification method, thereby enhancing the performance of query analysis.

[0057] The following Figure 4 The specific implementation methods of each step included in the illustrated embodiment are described in detail. This embodiment is applied to the overall framework after training:

[0058] In S401, a vector corresponding to the query statement is determined to obtain the query vector.

[0059] In this embodiment, the query statement to be queried is a query statement in the above query statement set, such as Figure 3 The query statement Q input in the inference phase is shown as i . Convert the query sentence into its corresponding vector representation to obtain the query vector.

[0060] In S402, the query vector is input into the trained first model, and a pseudo title vector is obtained according to the output of the first model, wherein the pseudo title vector includes a posteriori retrieval information of the query sentence.

[0061] In this embodiment, the first model is the above-mentioned simulation module, that is, Figure 3 The Imitation Module shown.

[0062] In the inference phase, the simulation module reconstructs the corresponding title representation according to the given query statement. Since the title representation is generated by the query and is different from the real title representation, it is named "pseudo title representation". For any of the two methods (detection method and generation method) in the simulation module, the pseudo title vector can be obtained after the query vector is calculated, that is, the pseudo title representation, such as Figure 3 The t shown ′ i . It can be considered that the pseudo title vector contains the posterior retrieval information of the query sentence.

[0063] In S403, the query sentence and the pseudo title vector are input into the trained second model, the target prediction of the query sentence is determined according to the output of the second model, and query analysis is performed based on the target prediction.

[0064] In this embodiment, the second model is the above-mentioned integrated module, that is, Figure 3 The integration module processes the query statement obtained with the pseudo title representation provided by the simulation module and makes a target prediction, that is, the predicted label category of the query statement, such as Figure 3 shown Query analysis can be performed based on target predictions.

[0065] The following introduces the training phases of the simulation module and the integration module in the PostIf framework respectively.

[0066] First, through Figure 5 and Figure 6 An embodiment of training a simulation module through a detection method is introduced.

[0067] In an exemplary embodiment, Figure 5 The flowchart of training the simulation module by the detection method is shown as an example. Figure 5, the embodiment shown in the figure includes S501-S505.

[0068] In S501, N training samples are obtained, where the i-th training sample includes the i-th query statement and the i-th title set corresponding to the i-th query statement, N is a positive integer, and i is a positive integer not greater than N.

[0069] refer to Figure 6 In this embodiment, a query set and its corresponding title set are input in the training phase. i , and its corresponding title set is If there are 5 query statements in the query statement set, take the first query statement in the set as an example, that is, N = 5, i = 1, then the first query statement is Q1, and its corresponding title set is

[0070] In S502, the i-th query statement is input into the first coding model to obtain the i-th query vector corresponding to the i-th query statement.

[0071] like Figure 6 As shown, the first encoding model is BERT query Query encoder. In the query encoder, a query sequence is added with a special marker "[CLS]" at the beginning and "[SEP]" at the end. The query statement is of the form: For example, a query statement: “What is a query in a database?”, its form in the query encoder is: {[CLS], What, is, a, query, in, a, database, [SEP]}.

[0072] In this embodiment, the i-th query statement is input into BERT query The encoder can get the corresponding i-th query vector. The input form is: q i =BERT query (Q i ).like Figure 6 As shown, q i That is, it is the i-th query vector. According to the above embodiment, when i=1, q1 is the first query vector.

[0073] In S503, the i-th title set is input into the second coding model to obtain the i-th title vector corresponding to the i-th title set.

[0074] like Figure 6 As shown, the second encoding model is BERT titleTitle encoder. Since there may be multiple titles associated with a query, in the title encoder, "[CLS]" and "[SEP]" are placed at the beginning and end of each title, respectively, to separate multiple titles.

[0075] Feed the i-th title set into BERT title The encoder can get the corresponding i-th title vector. The input format is: i =BERT title (T i ).like Figure 6 As shown, where t i According to the above embodiment, when i=1, t1 is the first title vector.

[0076] In S504, a first loss function of a first model is determined according to the i-th query vector and the i-th title vector.

[0077] In an exemplary embodiment, the loss function of the first model is the loss function of the simulation module in the detection method. The calculation method is:

[0078]

[0079] Among them, minimize Loss imi Indicates the calculation of the first loss function Loss imi Minimization, such as Figure 6 When i=1, the minimum loss function between the first query vector q1 and the first title vector t1 is calculated.

[0080] In S505, the parameters of the first coding model and the parameters of the second coding model are optimized according to the first loss function to determine the trained first model.

[0081] In this embodiment, during the training phase, the parameters of the two encoders in the simulation module are continuously optimized according to the first loss function to improve the similarity between the title and the query, and finally a trained simulation module is obtained and applied to the inference phase.

[0082] After the simulation module is trained using the detection method, the inference phase can begin.

[0083] In an exemplary embodiment, Figure 7 The flowchart of the query analysis method in another exemplary embodiment of the present disclosure is schematically shown. Specifically, Figure 7 is Figure 4 Among them, as a specific implementation of S402, S702 is based on Figure 5 The illustrated embodiment is implemented by a trained simulation module.

[0084] In this embodiment, step S702: input the query vector into the first coding model in the trained first model, and obtain the pseudo title vector according to the output of the first coding model.

[0085] According to the above description, the title corresponding to the query statement cannot be obtained in the inference stage. Therefore, corresponding to the training stage, it is only necessary to input the query statement into the query encoder to obtain the query vector corresponding to the query statement, and use this query vector as the pseudo title vector.

[0086] In summary, in the detection method, Figure 6 As shown, in the training phase, t i As the title representation, it is output and applied to the next module; in the inference stage, the query vector corresponding to the input query sentence is used as the pseudo title representation, output and applied to the next module.

[0087] The purpose of the detection method can be achieved through the above embodiments: the parameters of the encoder are optimized through the training phase, so that the simulation module can use the query representation to represent the title representation in the inference phase.

[0088] Next, through Figure 8 and Fig. 9 An embodiment of training a simulation module by a generative method is introduced.

[0089] In an exemplary embodiment, Figure 8 The flowchart of the training phase of the generation method is shown as an example. Figure 8 , the embodiment shown in the figure includes S801-S809.

[0090] In S801, N training samples are obtained, where the i-th training sample includes the i-th query statement and the i-th title set corresponding to the i-th query statement, N is a positive integer, and i is a positive integer not greater than N.

[0091] refer to Fig. 9 The structure diagram of the generation method shown in the figure shows that the overall structure uses a seq2seq (sequence-sequence) model, that is, an Encoder-Decoder (encoder-decoder) structure network. The encoder converts the input into a hidden state vector, which contains the content of the input sequence; the decoder performs the opposite process and converts the vector into an output sequence.

[0092] As in the training phase of the above detection method, a query set and its corresponding title set are input. Fig. 9 As shown, the i-th query statement is Q i , and its corresponding title set is

[0093] In S802, the i-th query statement is input into the third coding model to obtain a context vector corresponding to the i-th query statement.

[0094] In this embodiment, the third encoding model is BERT imi Encoder. Input the i-th query sentence into BERT imi After the encoder, the context vector corresponding to the i-th query statement can be obtained. The input form is: Q i =BERT imi (Q i ).like Fig. 9 As shown, Q i The context representation of the jth word in is the context vector. When i=1, j=1, Q1 is the context vector corresponding to the first query statement Q1. is the context vector of the first word in Q1.

[0095] In S803, a set of first hidden states corresponding to the context vector is obtained through an encoder in the sequence-to-sequence model.

[0096] In this embodiment, the sequence-sequence model is the above-mentioned seq2seq model. The encoder uses the LSTM (Long Short-Term Memory) network. en Encoder.

[0097] Feed the context vector into the LSTM en Encoder, the input form is:

[0098]

[0099] in, is the jth hidden state of the i-th query statement, that is, the first hidden state mentioned above, such as Fig. 9 As shown, When j = 0, Initialized to zero vector. When i=1, j=1, is the context vector of the first word in Q1, LSTM en Input You can get the first hidden state By analogy, the hidden state of the context vector corresponding to all query statements can be obtained, that is, the first hidden state.

[0100] In S804, the last hidden state of the first hidden state is used as the i-th pseudo title vector.

[0101] In this embodiment, the above-mentioned embodiment obtained The last hidden state in is taken as the i-th pseudo title vector t′ i ,Right now:

[0102]

[0103] In S805, a set of first probability vectors corresponding to the i-th title set is determined.

[0104] In this embodiment, the true probability vector of the jth word in the i-th title set is the first probability vector, such as Fig. 9 As shown, it is expressed as

[0105] In S806, the first hidden state and the first probability vector are input into a decoder in the sequence-sequence model, and a set of second hidden states is obtained according to the output of the decoder.

[0106] In this embodiment, the decoder in the sequence-sequence model uses LSTM de Decoder. The input format is:

[0107]

[0108] in, is the jth hidden state of the i-th title set, such as Fig. 9 As shown, First use the last hidden state in the first hidden state Initialize the decoder, that is, let All equals When j = 1, the input and are all zero vectors. When i=1,j=2, is the true probability vector of the second word in the first title set, LSTM de Input You can get the second hidden state of the first title set By analogy, the hidden state of all title sets, that is, the second hidden state, can be obtained.

[0109] In S807, the second hidden state is input into a normalized exponential function, and a set of second probability vectors is obtained according to the output of the normalized exponential function.

[0110] In this embodiment, the normalized exponential function is a Softmax function. The second probability vector is calculated by a fully connected layer activated by Softmax:

[0111]

[0112] in, is the predicted probability vector of the jth word in the i-th title set, i.e., the second probability vector, such as Fig. 9 As shown, W2 and b2 are the learnable weight matrix and bias vector respectively. When i=1, j=1, the input You can get the predicted probability vector of the first word in the first title set By analogy, the predicted probability vectors of all words, that is, the second probability vector, can be obtained.

[0113] In S808, a second loss function of the first model is determined according to a cross entropy function of the first probability vector and the second probability vector.

[0114] In this embodiment, the cross entropy function is used to measure the difference between the distribution of the pseudo title sequence and the real title sequence. The pseudo title sequence is the first probability vector The real title sequence is the second probability vector The second loss function is the loss function in the simulation module generation method. The method for calculating the second loss function is:

[0115]

[0116] Among them, minimize Loss′ imi Indicates the calculation of the second loss function Loss′ imi Minimization, such as Fig. 9 shown. is the first probability vector corresponding to the i-th title set, is the second probability vector corresponding to the i-th title set, CE(·) is the cross entropy function, and O is the length of the title set.

[0117] In S809, the parameters of the third encoding model and the parameters of the sequence-sequence model are optimized according to the second loss function to determine the trained first model.

[0118] In this embodiment, during the training phase, the parameters of the third encoding model and the sequence-sequence model in the simulation module are continuously optimized according to the second loss function to learn the conversion between the query and the title, and finally a trained simulation module is obtained and applied to the inference phase.

[0119] After the simulation module is trained through the generation method, it can enter the inference phase.

[0120] In an exemplary embodiment, Fig.10 The flowchart of the query analysis method in another exemplary embodiment of the present disclosure is schematically shown. Specifically, Fig.10 is Figure 4Among them, as a specific implementation of S402, S1002-S1004 are based on Figure 8 The embodiment shown is implemented by a trained simulation module.

[0121] In this embodiment, S1002: input the vector to be queried into the third encoding model in the trained first model to obtain a context vector corresponding to the vector to be queried.

[0122] According to the above description, the inference phase cannot obtain the title corresponding to the query sentence, so corresponding to the training phase, first input the query sentence into BERT imi In the encoder, the context vector corresponding to the query sentence is obtained.

[0123] In S1003, a set of first hidden states corresponding to the context vector is obtained through the encoder in the sequence-sequence model in the trained first model.

[0124] According to the training phase embodiment of the above generation method, the context vector is input into the LSTM en In the encoder, the first hidden state can be obtained.

[0125] In S1004, the last hidden state in a group of first hidden states is determined as a pseudo title vector.

[0126] In this embodiment, the last hidden state in the first hidden state drives the text sequence generation, so it can be used as a pseudo title representation generated by the simulation module.

[0127] In summary, in the generation method, Fig. 9 As shown, in both the training stage and the inference stage, the last hidden state in the first hidden state of the sample in each stage is used as the pseudo title representation.

[0128] The purpose of the generation method can be achieved through the above embodiments: the parameters of the encoder and the sequence-sequence model are optimized through the training phase, so that the simulation module can generate a pseudo title representation directly from the query under the guidance of the title during the inference phase.

[0129] When the simulation module completes training, the pseudo title of the output indicates the training for the integrated module. Fig.11 An embodiment of the training phase of the integration module is introduced.

[0130] In an exemplary embodiment, Fig.11 The flowchart of the integrated module training phase is shown as an example. Fig.11 , the embodiment shown in the figure includes S1101-S1108.

[0131] In S1101, the i-th label vector corresponding to the i-th query statement is obtained to obtain N training samples.

[0132] In this embodiment, reference Figure 3 The overall framework structure diagram shown in FIG. 1 needs to use the i-th query statement Q input in the training phase of the above simulation module. i The corresponding i-th label Y i The vector y i To calculate the loss function.

[0133] In S1102, the i-th pseudo title vector corresponding to the i-th query sentence obtained in the first model is obtained.

[0134] In this embodiment, if Figure 3 As shown, after the first model, that is, the simulation module training is completed, it is necessary to obtain the output t′ of the above simulation module training phase i , that is, the i-th pseudo title vector.

[0135] In S1103, embedding processing is performed on the i-th query statement to obtain the i-th embedding sequence.

[0136] refer to Figure 3 In this embodiment, after obtaining the i-th query statement, it is embedded. As described above, the i-th query statement Q i The form in the BERT encoder is: And according to the settings of BERT encoder, we get Q i The three embedding vectors of tag embedding, position embedding and segment embedding are added together to get the i-th embedding sequence in,

[0137] In S1104, the i-th pseudo title vector and the i-th embedded sequence are combined, and the combined vector is input into the fourth coding model.

[0138] refer to Figure 3 In this embodiment, the i-th pseudo title vector t′ i Append to the i-th embedded sequence In the above combined processing, we get the vector C i ,in And C i Input to BERT int Encoder, that is, in the fourth encoding model.

[0139] In S1105, the i-th target representation of the i-th query statement is obtained according to the output of the fourth encoding model.

[0140] Ci Enter BERT int The encoder is of the form: f i =BERT int (C i ). The final representation obtained is f i That is the above-mentioned i-th target representation, such as Figure 3 shown.

[0141] In S1106, the i-th target representation is input into a normalized exponential function, and the target prediction is obtained according to the output of the normalized exponential function.

[0142] The target prediction is calculated as:

[0143]

[0144] Among them, W1 and b1 are the learnable weight matrix and bias vector, respectively, and the i-th target representation f i Input together into the Softmax function fully connected layer fc (full connection), such as Figure 3 As shown. Get the query statement Q for the i-th query statement i Final prediction for the category That is the target prediction mentioned above.

[0145] In S1107, a third loss function of the second model is determined according to a cross entropy function between the i-th label vector and the target prediction.

[0146] In this embodiment, the third loss function is the loss function of the integrated module, and the calculation method is:

[0147]

[0148] Among them, minimize Loss int Indicates the calculation of the third loss function Loss int Minimization of For target prediction, is the i-th label vector, and there are G label vectors in total.

[0149] In S1108, the parameters of the fourth encoding model are optimized according to the third loss function to determine the trained second model.

[0150] In this embodiment, the loss function of the integrated module is calculated by the cross entropy of the final prediction and the true label vector, and BERT is optimized according to the loss function of the integrated module. int The parameters of the encoder are used to optimize the second model, the integration module.

[0151] In this technical solution, after completing the training of the simulation module and the integration module, the two modules will be trained as a whole. The training method is as follows:

[0152] First, a target loss function is determined according to the third loss function and the first loss function or the second loss function.

[0153] Among them, the calculation method of the final loss function, that is, the target loss function, is:

[0154] minimize Loss=Loss int +αLoss imi ,or

[0155] minimize Loss=Loss int +αLoss′ imi

[0156] Among them, minimize Loss means calculating the target loss function, Loss imi is the first loss function, Loss′ imi The second loss function, Loss int is the third loss function, and α is a hyperparameter. α controls the importance of the two modules. The larger the α, the more emphasis the PostIF framework places on the simulation module.

[0157] Secondly, the first model and the second model are optimized according to the objective loss function.

[0158] That is, the importance ratio of the simulation module and the integration module is controlled through the final loss function, and the simulation module and the integration module are optimized.

[0159] Through Fig.11 After the integrated module training is completed in the illustrated embodiment, the inference phase can be entered.

[0160] In an exemplary embodiment, Fig.12 The flowchart of a method for determining a target representation of a query statement is exemplified. Specifically, Fig.12 is Figure 4 As a specific implementation of step S403, S1201-S1204 are based on Fig.11 The embodiment shown is implemented by a trained integration module.

[0161] In S1201, the context vector corresponding to the query vector is embedded to obtain an embedded sequence.

[0162] In S1202, the pseudo title vector and the embedded sequence are combined and the combined vector is input into the fourth coding model.

[0163] In S1203, a target representation of the sentence to be queried is obtained according to the output of the fourth encoding model.

[0164] In S1204, the target representation is input into a normalized exponential function, and the target prediction is obtained according to the output of the normalized exponential function.

[0165] As with the embodiment of the integrated module training phase described above, the principles of the inference phase are the same except that the loss function does not need to be calculated, and will not be described in detail here.

[0166] The above-described embodiments can achieve the purpose of the integration module: the query information obtained from the input query is integrated with the title information provided by the simulation module, and a final prediction is made based on the information.

[0167] Referring to the query analysis results under different methods shown in Table 1, this experiment takes the query classification method in query analysis as an example, randomly selects 1 million query statements from real search logs and constructs a data set of the top five searched titles under the query. The experimental results are shown in Table 1:

[0168] Table 1

[0169]

[0170] As shown in Table 1, there are two data sets under the directory Catergory in the first column, namely the second-level category data set Second-level and the first-level category data set First-level. In the second column, for each data set, eight query analysis methods Method are given, including six original baseline methods, namely: BERT, ERNIE, BERT-Finetune, Feature Transfer, PFD and BERT-Title, and the remaining two are the methods proposed by this technical solution: PostIF-Detective (detection method) and PostIF-Generative (generation method). In the third column, the experimental results on the validation data set Validation Set are listed; in the fourth column, the experimental results on the test data set Test Set are listed. For each data set, there are four evaluation indicators, namely: Accuracy, Precision, Recall and F1 score F1-score.

[0171] According to the experimental results, the query analysis performance has been significantly improved by using the PostIF framework of this technical solution. With the introduction and integration of posterior retrieval information, the classifier performance (accuracy, precision, recall, F1 score) has been further improved compared with previous query classification methods.

[0172] In an exemplary embodiment, Fig.13 Schematically shows a structural diagram of a query analysis system according to another exemplary embodiment of the present disclosure. Fig.13 :

[0173] In an exemplary embodiment, the query analysis system 1300 includes: a conversion module 1301 , a simulation module 1302 , and an integration module 1303 .

[0174] The conversion module 1301 is used to: determine the vector corresponding to the query statement to obtain the query vector;

[0175] The simulation module 1302 is used to: input the query vector into the first model, and obtain a pseudo title vector according to the output of the first model, wherein the pseudo title vector includes the a posteriori retrieval information of the query sentence;

[0176] The integration module 1303 is used to: input the query sentence and the pseudo title vector into the second model, determine the target prediction of the query sentence according to the output of the second model, and perform query analysis based on the target prediction.

[0177] It should be noted that when the data synchronization system provided in the above embodiment executes the query analysis method, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the query analysis system and the query analysis method embodiment provided in the above embodiment belong to the same concept. Therefore, for details not disclosed in the embodiment of the system disclosed in the present invention, please refer to the embodiment of the query analysis method disclosed in the present invention, which will not be repeated here.

[0178] The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0179] The embodiments of the present disclosure also provide a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the method of any of the above embodiments are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0180] The embodiment of the present disclosure also provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the program.

[0181] Fig.14 The structure diagram of the terminal in an exemplary embodiment of the present disclosure is schematically shown. Fig.14 As shown, the terminal 1400 includes: a processor 1401 and a memory 1402 .

[0182] In the embodiment of the present disclosure, the processor 1401 is the control center of the computer system, which can be the processor of a physical machine or the processor of a virtual machine. The processor 1401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1401 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0183] In the embodiment of the present disclosure, the processor 1401 is specifically used for:

[0184] Determine a vector corresponding to a sentence to be queried and obtain the vector to be queried; input the vector to be queried into a trained first model, and obtain a pseudo title vector according to the output of the first model, wherein the pseudo title vector includes a posteriori retrieval information of the sentence to be queried; input the sentence to be queried and the pseudo title vector into a trained second model, determine a target prediction of the sentence to be queried according to the output of the second model, and perform query analysis based on the target prediction.

[0185] Further, in an embodiment of the present disclosure, the first model includes a first encoding model and a second encoding model, and the method includes: obtaining N training samples, the i-th training sample includes an i-th query statement and an i-th title set corresponding to the i-th query statement, N is a positive integer, and i is a positive integer not greater than N; inputting the i-th query statement into the first encoding model to obtain an i-th query vector corresponding to the i-th query statement; inputting the i-th title set into the second encoding model to obtain an i-th title vector corresponding to the i-th title set; determining a first loss function of the first model according to the i-th query vector and the i-th title vector; optimizing parameters of the first encoding model and parameters of the second encoding model according to the first loss function to determine the trained first model.

[0186] Optionally, the first loss function calculation method is: Among them, minimize Lossimi means calculating the above first loss function Loss imi Minimize q i is the i-th query vector mentioned above, t i is the i-th title vector mentioned above.

[0187] Optionally, the above-mentioned inputting the above-mentioned query vector into the trained first model and obtaining the pseudo title vector according to the output of the above-mentioned first model includes: inputting the above-mentioned query vector into the first encoding model in the above-mentioned trained first model, and obtaining the above-mentioned pseudo title vector according to the output of the above-mentioned first encoding model.

[0188] Optionally, the first model includes a third encoding model and a sequence-sequence model, and the method further includes: obtaining N training samples, the i-th training sample includes the i-th query statement and the i-th title set corresponding to the i-th query statement, N is a positive integer, and i is a positive integer not greater than N; inputting the i-th query statement into the third encoding model to obtain a context vector corresponding to the i-th query statement; obtaining a group of first hidden states corresponding to the context vector through the encoder in the sequence-sequence model; using the last hidden state of the first hidden state as the i-th pseudo title vector corresponding to the i-th query statement; determining a group of first probability vectors corresponding to the i-th title set; inputting the first hidden state and the first probability vector into the decoder in the sequence-sequence model to obtain a group of second hidden states according to the output of the decoder; inputting the second hidden state into a normalized exponential function to obtain a group of second probability vectors according to the output of the normalized exponential function; determining the second loss function of the first model according to the cross entropy function of the first probability vector and the second probability vector; optimizing the parameters of the third encoding model and the parameters of the sequence-sequence model according to the second loss function to determine the trained first model.

[0189] Optionally, the second loss function calculation method is: Among them, minimizeLoss′ imi Indicates the calculation of the second loss function Loss′ imi Minimization of is a set of first probability vectors corresponding to the above i-th title set, is a set of second probability vectors corresponding to the above i-th title set, CE(·) is the above cross entropy function, O is the length of all the above title sets, O is a positive integer, and l is a positive integer not greater than O.

[0190] Optionally, the above-mentioned inputting the above-mentioned query vector into the trained first model and obtaining the pseudo title vector according to the output of the above-mentioned first model includes: inputting the above-mentioned query vector into the third encoding model in the above-mentioned trained first model to obtain the context vector corresponding to the above-mentioned query vector; obtaining a group of first hidden states corresponding to the above-mentioned context vector through the encoder in the sequence-sequence model in the above-mentioned trained first model; and determining the last hidden state in the above-mentioned group of first hidden states as the above-mentioned pseudo title vector.

[0191] Optionally, the method further includes: obtaining the i-th label vector corresponding to the i-th query statement to obtain N training samples; obtaining the i-th pseudo title vector corresponding to the i-th query statement obtained in the first model; embedding the i-th query statement to obtain the i-th embedding sequence; combining the i-th pseudo title vector and the i-th embedding sequence, and inputting the combined vector into a fourth encoding model; obtaining the i-th target representation of the i-th query statement according to the output of the fourth encoding model; inputting the i-th target representation into the normalized exponential function, and obtaining the target prediction according to the output of the normalized exponential function; determining the third loss function of the second model according to the cross entropy function of the i-th label vector and the target prediction; optimizing the parameters of the fourth encoding model according to the third loss function to determine the trained second model.

[0192] Optionally, the third loss function calculation method is: Among them, minimizeLossint means to calculate the minimization of the third loss function Lossint mentioned above. To predict the above goals, is the i-th label vector mentioned above, there are G label vectors mentioned above, G is a positive integer, and j is a positive integer not greater than G.

[0193] Optionally, the method further includes: determining a target loss function based on the third loss function and the first loss function or the second loss function; and optimizing the first model and the second model based on the target loss function.

[0194] Optionally, the above objective loss function calculation method is: minimizeLoss = Loss int +αLoss imi , or minimizeLoss = Loss int +αLoss′ imi , where minimizeLoss means calculating the above target loss function, Loss imi is the first loss function, Loss′ imi The second loss function mentioned above, Loss int is the third loss function mentioned above, and α is a hyperparameter.

[0195] Optionally, the above-mentioned input of the sentence to be queried and the above-mentioned pseudo title vector into the trained second model, and determining the target prediction of the sentence to be queried according to the output of the above-mentioned second model, including: embedding the context vector corresponding to the above-mentioned vector to be queried to obtain an embedding sequence; combining the above-mentioned pseudo title vector and the above-mentioned embedding sequence, and inputting the combined vector into a fourth encoding model; obtaining the target representation of the sentence to be queried according to the output of the above-mentioned fourth encoding model; inputting the above-mentioned target representation into the above-mentioned normalized exponential function, and obtaining the above-mentioned target prediction according to the output of the above-mentioned normalized exponential function.

[0196] The memory 1402 may include one or more computer-readable storage media, which may be non-transitory. The memory 1402 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments of the present disclosure, the non-transitory computer-readable storage medium in the memory 1402 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1401 to implement the method in the embodiment of the present disclosure.

[0197] In some embodiments, the terminal 1400 further includes: a peripheral device interface 1403 and at least one peripheral device. The processor 1401, the memory 1402 and the peripheral device interface 1403 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1403 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a display screen 1404, a camera 1405 and an audio circuit 14014.

[0198] The peripheral device interface 1403 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1401 and the memory 1402. In some embodiments of the present disclosure, the processor 1401, the memory 1402, and the peripheral device interface 1403 are integrated on the same chip or circuit board; in some other embodiments of the present disclosure, any one or two of the processor 1401, the memory 1402, and the peripheral device interface 1403 may be implemented on a separate chip or circuit board. The embodiments of the present disclosure are not specifically limited to this.

[0199] The display screen 1404 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1404 is a touch display screen, the display screen 1404 also has the ability to collect touch signals on the surface or above the surface of the display screen 1404. The touch signal can be input to the processor 1401 as a control signal for processing. At this time, the display screen 1404 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments of the present disclosure, the display screen 1404 can be one, and the front panel of the terminal 1400 is set; in other embodiments of the present disclosure, the display screen 1404 can be at least two, which are respectively set on different surfaces of the terminal 1400 or are folded; in some other embodiments of the present disclosure, the display screen 1404 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 1400. Even, the display screen 1404 can also be set as a non-rectangular irregular figure, that is, a special-shaped screen. The display screen 1404 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0200] Camera 1405 is used to capture images or videos. Optionally, camera 1405 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments of the present disclosure, camera 1405 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0201] The audio circuit 1406 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1401 for processing. For the purpose of stereo sound collection or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1400. The microphone may also be an array microphone or an omnidirectional collection microphone.

[0202] The power supply 1407 is used to power various components in the terminal 1400. The power supply 1407 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1407 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0203] The terminal structure block diagram shown in the embodiment of the present disclosure does not constitute a limitation on the terminal 1400. The terminal 1400 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0204] In the present disclosure, the terms "first", "second", etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or order; the term "plurality" refers to two or more, unless otherwise expressly defined. The terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; "connected" can be a direct connection or an indirect connection through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to the specific circumstances.

[0205] In the description of the present disclosure, it is necessary to understand that the directions or positional relationships indicated by the terms "upper" and "lower" etc. are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the system or unit referred to must have a specific direction, be constructed and operated in a specific orientation, and therefore, should not be understood as a limitation on the present disclosure.

[0206] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, equivalent changes made according to the claims of the present disclosure are still within the scope covered by the present disclosure.

Claims

1. A query analysis method, characterized in that: include: Determine the vector corresponding to the query statement to obtain the query vector; Inputting the query vector into the trained first model, and obtaining a pseudo title vector according to the output of the first model, wherein the pseudo title vector includes a posteriori retrieval information of the query sentence; Inputting the query sentence and the pseudo title vector into the trained second model, determining a target prediction of the query sentence according to an output of the second model, and performing query analysis based on the target prediction; The first model includes a third encoding model and a sequence-sequence model, and the method further includes: Obtain N training samples, where the i-th training sample includes the i-th query statement and the i-th title set corresponding to the i-th query statement, where N is a positive integer and i is a positive integer not greater than N; Inputting the i-th query statement into the third encoding model to obtain a context vector corresponding to the i-th query statement; Obtaining, through an encoder in the sequence-to-sequence model, a set of first hidden states corresponding to the context vector; Using the last hidden state of the first hidden state as the i-th pseudo title vector corresponding to the i-th query statement; Determine a set of first probability vectors corresponding to the i-th title set; Inputting the first hidden state and the first probability vector into a decoder in the sequence-sequence model, and obtaining a set of second hidden states according to an output of the decoder; Inputting the second hidden state into a normalized exponential function, and obtaining a set of second probability vectors according to an output of the normalized exponential function; Determining a second loss function of the first model according to a cross entropy function of the first probability vector and the second probability vector; Optimize the parameters of the third encoding model and the parameters of the sequence-sequence model according to the second loss function to determine the trained first model.

2. The query analysis method according to claim 1, characterized in that: The first model includes a first coding model and a second coding model, and the method includes: Inputting the i-th query statement into a first encoding model to obtain an i-th query vector corresponding to the i-th query statement; Inputting the i-th title set into a second coding model to obtain an i-th title vector corresponding to the i-th title set; Determine a first loss function of the first model according to the i-th query vector and the i-th title vector; Optimize the parameters of the first encoding model and the parameters of the second encoding model according to the first loss function to determine the trained first model.

3. The query analysis method according to claim 2, characterized in that: The first loss function calculation method is: Among them, minimize Loss imi Indicates the calculation of the first loss function Loss imi Minimize q i is the i-th query vector, t i is the i-th title vector.

4. The query analysis method according to claim 2, characterized in that: The step of inputting the query vector into the trained first model and obtaining a pseudo title vector according to an output of the first model includes: The query vector is input into a first encoding model in the trained first model, and the pseudo title vector is obtained according to an output of the first encoding model.

5. The query analysis method according to claim 1, characterized in that: The second loss function calculation method is: Among them, minimize Loss′ imi Indicates the calculation of the second loss function Loss′ imi Minimization of is a set of first probability vectors corresponding to the i-th title set, is a set of second probability vectors corresponding to the i-th title set, CE(·) is the cross entropy function, O is the length of all the title sets, O is a positive integer, and l is a positive integer not greater than O.

6. The query analysis method according to claim 1, characterized in that: The step of inputting the query vector into the trained first model and obtaining a pseudo title vector according to an output of the first model includes: Inputting the query vector into the third encoding model in the trained first model to obtain a context vector corresponding to the query vector; Obtaining a set of first hidden states corresponding to the context vector through an encoder in a sequence-sequence model in the trained first model; The last hidden state in the group of first hidden states is determined as the pseudo title vector.

7. The query analysis method according to claim 1 or 2, characterized in that: The method further comprises: Obtaining the i-th label vector corresponding to the i-th query statement to obtain N training samples; Performing embedding processing on the i-th query statement to obtain an i-th embedding sequence; Obtaining an i-th pseudo title vector corresponding to the i-th query statement obtained in the first model; Combining the i-th pseudo title vector and the i-th embedded sequence, and inputting the combined vector into a fourth coding model; Obtaining an i-th target representation of the i-th query statement according to an output of the fourth encoding model; Inputting the i-th target representation into the normalized exponential function, and obtaining the target prediction according to the output of the normalized exponential function; Determining a third loss function of the second model according to the cross entropy function of the i-th label vector and the target prediction; Optimize the parameters of the fourth encoding model according to the third loss function to determine the trained second model.

8. The query analysis method according to claim 7, characterized in that: The third loss function calculation method is: Among them, minimize Loss int Indicates the calculation of the third loss function Loss int Minimization of For the target prediction, is the i-th label vector, there are L label vectors in total, G is a positive integer, and j is a positive integer not greater than G.

9. The query analysis method according to claim 7, characterized in that: The method further comprises: Determine a target loss function according to the third loss function and the first loss function or the second loss function; The first model and the second model are optimized according to the objective loss function.

10. The query analysis method according to claim 9, characterized in that: The objective loss function calculation method is: minimize Loss=Loss int +αLoss imi or minimize Loss=Loss int +αLoss′ imi Among them, minimize Loss means calculating the target loss function, Loss imi is the first loss function, Loss′ imi The second loss function, Loss int is the third loss function, and α is a hyperparameter.

11. The query analysis method according to claim 7, characterized in that: The step of inputting the query sentence and the pseudo title vector into a trained second model, and determining a target prediction of the query sentence according to an output of the second model, comprises: Embedding the context vector corresponding to the query vector to obtain an embedding sequence; Combining the pseudo title vector with the embedded sequence, and inputting the combined vector into a fourth coding model; Obtaining a target representation of the sentence to be queried according to an output of the fourth encoding model; The target representation is input into the normalized exponential function, and the target prediction is obtained according to the output of the normalized exponential function.

12. A query analysis system, characterized in that: include: A conversion module is used to: determine a vector corresponding to the query statement to obtain a query vector; A simulation module, used to: input the query vector into a first model, and obtain a pseudo title vector according to the output of the first model, wherein the pseudo title vector includes a posteriori retrieval information of the query sentence, and the first model includes a third encoding model and a sequence-sequence model; An integration module, used to: input the query sentence and the pseudo title vector into a second model, determine a target prediction of the query sentence according to an output of the second model, and perform query analysis based on the target prediction; A model training module, used to obtain N training samples, the i-th training sample includes the i-th query statement and the i-th title set corresponding to the i-th query statement, N is a positive integer, and i is a positive integer not greater than N; Inputting the i-th query statement into the third encoding model to obtain a context vector corresponding to the i-th query statement; Obtaining, through an encoder in the sequence-to-sequence model, a set of first hidden states corresponding to the context vector; Using the last hidden state of the first hidden state as the i-th pseudo title vector corresponding to the i-th query statement; Determine a set of first probability vectors corresponding to the i-th title set; Inputting the first hidden state and the first probability vector into a decoder in the sequence-sequence model, and obtaining a set of second hidden states according to an output of the decoder; Inputting the second hidden state into a normalized exponential function, and obtaining a set of second probability vectors according to an output of the normalized exponential function; Determining a second loss function of the first model according to a cross entropy function of the first probability vector and the second probability vector; Optimize the parameters of the third encoding model and the parameters of the sequence-sequence model according to the second loss function to determine the trained first model.

13. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the query analysis method according to any one of claims 1 to 11 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the query analysis method according to any one of claims 1 to 11 is implemented.

Images