A keyword intelligent identification method and system based on user behavior characteristics

Abstract

Disclosed is a method and system for intelligently distinguishing keywords based on user behavior characteristics. First, the user search behavior data is subjected to text preprocessing and word segmentation processing, and then a text convolutional neural network model is used to obtain a user search behavior semantic understanding feature vector. Next, the search query input by the analyzed user is subjected to word segmentation processing, and then a word embedding layer is used to obtain a sequence of search word embedding vectors. Then, the sequence of search word embedding vectors is passed through a context encoder to obtain a sequence of context search word feature vectors. Next, the semantic association matrix between each context search word feature vector and the user search behavior semantic understanding feature vector is calculated to obtain multiple semantic association matrices. Finally, the multiple semantic association matrices are respectively passed through a classifier to obtain multiple probability values, and the search word corresponding to the largest of the multiple probability values ​​is used as a keyword. In this way, keywords can be intelligently distinguished.

Description

Technical Field

[0001] The present application relates to the field of keyword identification, and more specifically, to a method and system for intelligent keyword identification based on user behavior characteristics. Background Art

[0002] Keyword resolution refers to the screening and classification of keywords extracted from a text in order to better understand, summarize and use these keywords. Keyword recognition based on text content refers to the automatic identification of keywords that best represent the theme or meaning of the text. Although it has a wide range of applications in natural language processing and information retrieval, it also has problems such as difficulty in resolving polysemous words, long-tail word processing, noise interference, and data sparsity.

[0003] Search behavior data refers to the relevant data generated when users search on search engines or other platforms. This data usually includes information such as search keywords, search time, search results, clicked links and dwell time. It can often reflect their real needs and interests without being interfered by text information such as titles and summaries.

[0004] Therefore, we look forward to an intelligent keyword identification solution based on user behavior characteristics. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method and system for intelligently distinguishing keywords based on user behavior characteristics. It first performs text preprocessing on the user search behavior data and performs word segmentation processing, and then passes through a text convolutional neural network model to obtain a user search behavior semantic understanding feature vector. Then, the search query input by the analyzed user is processed by word segmentation and then passed through a word embedding layer to obtain a sequence of search word embedding vectors. Then, the sequence of search word embedding vectors is passed through a context encoder to obtain a sequence of context search word feature vectors. Then, the semantic association matrix between each context search word feature vector and the user search behavior semantic understanding feature vector is calculated to obtain multiple semantic association matrices. Finally, the multiple semantic association matrices are respectively passed through a classifier to obtain multiple probability values ​​and the search word corresponding to the largest of the multiple probability values ​​is used as a keyword. In this way, keywords can be intelligently distinguished.

[0006] According to one aspect of the present application, a method for intelligent keyword identification based on user behavior characteristics is provided, which includes: obtaining search behavior data of the analyzed user; performing text preprocessing on the user search behavior data to obtain preprocessed user search behavior data; performing word segmentation processing on the preprocessed user search behavior data and passing it through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector; obtaining a search query input by the analyzed user; performing word segmentation processing on the search query and passing it through a word embedding layer to obtain a sequence of search word embedding vectors; passing the sequence of search word embedding vectors through a converter-based context encoder to obtain a sequence of context search word feature vectors; calculating the semantic association matrix between each context search word feature vector in the sequence of context search word feature vectors and the user search behavior semantic understanding feature vector to obtain multiple semantic association matrices; passing the multiple semantic association matrices through a classifier respectively to obtain multiple probability values; and taking the search word corresponding to the largest of the multiple probability values ​​as a keyword.

[0007] In the above-mentioned keyword intelligent identification method based on user behavior characteristics, the preprocessed user search behavior data is segmented and then passed through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector, including: performing word segmentation on the preprocessed user search behavior data to convert the preprocessed user search behavior data into a word sequence composed of multiple words; using the word embedding layer of the text convolutional neural network model to map each word in the word sequence to a word vector to obtain a sequence of word embedding vectors; and passing the sequence of word embedding vectors through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector.

[0008] In the above-mentioned keyword intelligent identification method based on user behavior characteristics, the sequence of word embedding vectors is passed through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector, including: using each layer of the text convolutional neural network model to perform convolution processing, mean pooling processing and non-linear activation processing on the input data in the forward pass of the layer to output the user search behavior semantic understanding feature vector from the last layer of the text convolutional neural network model, wherein the input of the first layer of the text convolutional neural network model is the sequence of word embedding vectors.

[0009] In the above-mentioned keyword intelligent identification method based on user behavior characteristics, the sequence of search term embedding vectors is passed through a converter-based context encoder to obtain a sequence of context search term feature vectors, including: one-dimensionally arranging the sequence of search term embedding vectors to obtain a global word sequence feature vector; calculating the product between the global word sequence feature vector and the transposed vector of each search term embedding vector in the sequence of search term embedding vectors to obtain multiple self-attention association matrices; respectively normalizing each of the multiple self-attention association matrices to obtain multiple standardized self-attention association matrices; passing each of the multiple standardized self-attention association matrices through a Softmax classification function to obtain multiple probability values; and weighting each of the search term embedding vectors in the sequence of search term embedding vectors using each of the multiple probability values ​​as a weight to obtain the sequence of context search term feature vectors.

[0010] In the above-mentioned keyword intelligent identification method based on user behavior characteristics, the semantic association matrix between each context retrieval word feature vector in the sequence of the context retrieval word feature vector and the user search behavior semantic understanding feature vector is calculated to obtain multiple semantic association matrices, including: calculating the semantic association matrix between each context retrieval word feature vector in the sequence of the context retrieval word feature vector and the user search behavior semantic understanding feature vector by the following association formula to obtain the multiple semantic association matrices; wherein the association formula is: ,in, represents each context retrieval word feature vector in the sequence of context retrieval word feature vectors, represents the transposed vector of a vector, represents the semantic understanding feature vector of the user search behavior, represents each semantic association matrix in the plurality of semantic association matrices, Represents vector multiplication.

[0011] In the above-mentioned keyword intelligent identification method based on user behavior characteristics, it also includes a training step: training the text convolutional neural network model including the word embedding layer, the converter-based context encoder and the classifier; wherein the training step includes: obtaining the training search behavior data of the analyzed user and the true value of the keyword; performing text preprocessing on the training search behavior data to obtain the user search behavior data after training preprocessing; performing word segmentation processing on the user search behavior data after training preprocessing and passing it through the text convolutional neural network model including the word embedding layer to obtain the training user search behavior semantic understanding feature vector; obtaining the training search query input by the analyzed user; performing word segmentation processing on the training search query and passing it through the word embedding layer to obtain a sequence of training search word embedding vectors; embedding the sequence of training search word embedding vectors A sequence of training context retrieval term feature vectors is obtained through the converter-based context encoder; a semantic association matrix between each training context retrieval term feature vector in the sequence of training context retrieval term feature vectors and the training user search behavior semantic understanding feature vector is calculated to obtain a plurality of training semantic association matrices; a streaming refinement loss function value of each training context retrieval term feature vector and the training user search behavior semantic understanding feature vector is calculated; the plurality of training semantic association matrices are respectively passed through the classifier to obtain a plurality of classification loss function values; and a weighted sum of the streaming refinement loss function value and the plurality of classification loss function values ​​is used as a loss function, and the text convolutional neural network model including a word embedding layer, the converter-based context encoder and the classifier are trained through directional propagation of gradient descent.

[0012] In the above-mentioned keyword intelligent identification method based on user behavior characteristics, the streaming refinement loss function value of each training context retrieval word feature vector and the training user search behavior semantic understanding feature vector is calculated, including: calculating the streaming refinement loss function value of each training context retrieval word feature vector and the training user search behavior semantic understanding feature vector using the following loss formula; ,in, represents the feature vectors of each training context retrieval word, represents the semantic understanding feature vector of the training user search behavior, represents the square of the vector's two-norm, and and represent position-wise subtraction and multiplication of vectors, respectively. represents the exponential operation of a vector, wherein the exponential operation of a vector represents the calculation of the value of a natural exponential function raised to the power of the eigenvalues ​​at each position in the vector, Represents the streaming refinement loss function value. According to another aspect of the present application, a keyword intelligent resolution system based on user behavior characteristics is provided, which includes: a search behavior data acquisition module for acquiring search behavior data of the analyzed user; a text preprocessing module for performing text preprocessing on the user search behavior data to obtain preprocessed user search behavior data; an embedding convolutional coding module for performing word segmentation processing on the preprocessed user search behavior data and then passing it through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector; a user input data acquisition module for acquiring a search query input by the analyzed user; an embedding module for performing word segmentation processing on the search query and then passing it through a word embedding layer to obtain a sequence of search word embedding vectors; a context encoding module for passing the sequence of search word embedding vectors through a converter-based context encoder to obtain a sequence of context search word feature vectors; a semantic association module for calculating a semantic association matrix between each context search word feature vector in the sequence of context search word feature vectors and the user search behavior semantic understanding feature vector to obtain multiple semantic association matrices; a classification module for passing the multiple semantic association matrices through a classifier respectively to obtain multiple probability values; and

[0013] The keyword acquisition module is used to use the search term corresponding to the largest one among the multiple probability values ​​as the keyword.

[0014] In the above-mentioned keyword intelligent resolution system based on user behavior characteristics, the embedded convolutional coding module is used to: perform word segmentation processing on the preprocessed user search behavior data to convert the preprocessed user search behavior data into a word sequence composed of multiple words; use the word embedding layer of the text convolutional neural network model to map each word in the word sequence to a word vector to obtain a sequence of word embedding vectors; and pass the sequence of word embedding vectors through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector.

[0015] In the above-mentioned keyword intelligent resolution system based on user behavior characteristics, the sequence of word embedding vectors is passed through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector, including: using each layer of the text convolutional neural network model to perform convolution processing, mean pooling processing and non-linear activation processing on the input data in the forward pass of the layer so that the last layer of the text convolutional neural network model outputs the user search behavior semantic understanding feature vector, wherein the input of the first layer of the text convolutional neural network model is the sequence of word embedding vectors.

[0016] Compared with the prior art, the method and system for intelligent keyword identification based on user behavior characteristics provided by the present application first pre-processes the user search behavior data and performs word segmentation processing, and then passes through a text convolutional neural network model to obtain a user search behavior semantic understanding feature vector, then, the search query input by the analyzed user is segmented and passed through a word embedding layer to obtain a sequence of search word embedding vectors, then, the sequence of search word embedding vectors is passed through a context encoder to obtain a sequence of context search word feature vectors, then, the semantic association matrix between each context search word feature vector and the user search behavior semantic understanding feature vector is calculated to obtain multiple semantic association matrices, and finally, the multiple semantic association matrices are respectively passed through a classifier to obtain multiple probability values ​​and the search word corresponding to the largest of the multiple probability values ​​is used as a keyword. In this way, keywords can be intelligently identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. The following drawings are not deliberately scaled according to the actual size, and the focus is on illustrating the main purpose of the present application.

[0018] Figure 1 This is an application scenario diagram of the keyword intelligent identification method based on user behavior characteristics according to an embodiment of the present application.

[0019] Figure 2 The present invention is a flowchart of a method for intelligently identifying keywords based on user behavior characteristics according to an embodiment of the present application.

[0020] Figure 3 Schematic diagram of the architecture of a method for intelligently identifying keywords based on user behavior characteristics according to an embodiment of the present application.

[0021] Figure 4 This is a flowchart of sub-step S130 of the keyword intelligent identification method based on user behavior characteristics according to an embodiment of the present application.

[0022] Figure 5 This is a flowchart of sub-step S160 of the keyword intelligent identification method based on user behavior characteristics according to an embodiment of the present application.

[0023] Figure 6 The flowchart is a training step further included in the keyword intelligent identification method based on user behavior characteristics according to an embodiment of the present application.

[0024] Figure 74 is a block diagram of a keyword intelligent identification system based on user behavior characteristics according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0026] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0027] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0028] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0029] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0030] In response to the above technical problems, the technical concept of this application is: since the user's search and browsing behavior can often reflect their real needs and interests, and is not interfered by text information such as titles and abstracts, it is expected to introduce user behavior characteristics into the keyword resolution scheme. Specifically, by obtaining the search behavior data of the analyzed user and the search query input by the analyzed user, and combining deep learning and artificial intelligence technology to extract keywords.

[0031] Specifically, in the technical solution of this application, first, the search behavior data of the user to be analyzed is obtained. As mentioned before, the search and browsing behaviors of users can often reflect their true needs and interests, without being interfered by text information such as titles and abstracts. Compared with traditional keyword recognition methods based on text content, the method based on user behavior can more accurately reflect the user's hobbies and behavior habits. In addition, the method based on user behavior can also characterize and analyze users at a finer granularity.

[0032] Considering that the user search behavior data usually contains a lot of useless or duplicate information, which has no practical effect on keyword discrimination and information understanding, in the technical solution of this application, the user search behavior data is preprocessed to obtain preprocessed user search behavior data. For example, during text preprocessing, it can be cleared and optimized by removing stop words and performing stemming, etc., so as to retain useful information and improve the accuracy and efficiency of the model. Specifically, stop words refer to some words that often appear in the language but have no practical meaning in the text, such as "de" (of), "le" (particle indicating completion), etc., which can be removed during data processing. Stemming refers to converting different forms of words (such as different tenses of verbs, different plurals of nouns, etc.) into their basic forms, such as converting "running", "runs", "ran", etc. into "run", which can avoid the interference of different forms of words on the model.

[0033] Since the preprocessed user search behavior data is text data, in the technical solution of this application, after the preprocessed user search behavior data is tokenized, it is passed through a text convolutional neural network model including an embedding layer to convert the text into a numerical representation and extract semantic feature information from it, thereby obtaining a user search behavior semantic understanding feature vector. Here, the embedding layer can map each word to a vector in a high-dimensional space, thereby retaining the semantic relationship between words; the text convolutional neural network model uses operations such as convolutional layers and pooling layers to extract features from the sequence of embedding vectors, thereby obtaining a feature vector that can represent the user's search behavior intention.

[0034] In order to obtain the user's current search needs and convert them into a numerical representation for easy comparison and matching with their historical search behaviors, in the technical solution of this application, first, the retrieval query input by the user to be analyzed is obtained; then, after the retrieval query is tokenized, it is passed through the embedding layer to obtain a sequence of retrieval word embedding vectors.

[0035] Considering that there is a contextual relationship between search terms, if this contextual relationship can be captured, the effect of subsequent classification can be greatly improved. In the technical solution of the present application, the sequence of search term embedding vectors is passed through a transformer-based context encoder to obtain a sequence of contextual search term feature vectors. Here, the transformer-based context encoder uses a self-attention mechanism to capture the correlation between each element in the sequence and the global semantic information as the background of each element.

[0036] Then, a semantic association matrix between each context search term feature vector in the sequence of context search term feature vectors and the user search behavior semantic understanding feature vector is calculated to measure the semantic similarity between the search term and the user search behavior, thereby obtaining multiple semantic association matrices.

[0037] Then, the multiple semantic association matrices are respectively passed through a classifier to obtain multiple probability values, and the search term corresponding to the largest of the multiple probability values ​​is used as a keyword. The classifier is a neural network model, whose input is a semantic association matrix, and whose output is a probability value, indicating the possibility of whether the search term is a keyword. In the technical solution of the present application, by selecting the search term corresponding to the largest probability value as a keyword, the keyword that best represents the user's needs and interests can be selected.

[0038] Here, when calculating the semantic association matrix between each context search term feature vector in the sequence of the context search term feature vector and the user search behavior semantic understanding feature vector to obtain multiple semantic association matrices, what is essentially done is to map the serialized context encoding features of the search term represented by each context search term feature vector and the textual sequence semantic feature representation of the user search behavior represented by the user search behavior semantic understanding feature vector into a high-dimensional association feature space. Therefore, if the correlation between the context search term feature vector and the user search behavior semantic understanding feature vector in the association space dimension between the serialized textual semantic feature representation and the high-dimensional features can be improved, the expression effect of the semantic association matrix can be improved.

[0039] Based on this, the applicant of the present application further introduces a classification loss function based on the semantic association matrix for each context search word feature vector. and the user search behavior semantic understanding feature vector The streaming refinement loss function is expressed as: in Represents the square of the two-norm of a vector.

[0040] Here, the streaming refinement loss function is based on each of the context retrieval word feature vectors and the user search behavior semantic understanding feature vector In the conversion of the serialized encoded streaming distribution of the text to the spatial distribution in the high-dimensional associated feature space, the super-resolution improvement of the spatial distribution in the high-dimensional associated feature space is achieved by synchronously performing interpolation under the sequence distribution of the vector, so as to provide a more refined alignment of the distribution differences in the high-dimensional associated feature space by balancing the mutual probability relationship under the sequence, so as to jointly present the cross-dimensional (inter-dimensional) contextual associations in the serialized text semantic feature dimension and the associated feature space dimension, thereby improving the expression effect of the multiple semantic association matrices, and further improving the accuracy of the multiple probability values ​​obtained by the multiple semantic association matrices through the classifier.

[0041] This application has the following technical effects: 1. It provides a keyword intelligent identification solution based on user behavior characteristics.

[0042] 2. The above-mentioned intelligent keyword identification solution based on user behavior characteristics can more accurately grasp the real needs and interests of users through analysis and understanding of user search behavior characteristics, thereby improving the accuracy of keyword identification.

[0043] Figure 1 FIG. 1 is an application scenario diagram of a keyword intelligent identification method based on user behavior characteristics according to an embodiment of the present application. Figure 1 As shown, in this application scenario, first, the search behavior data of the analyzed user is obtained (for example, Figure 1 D1) and the search query entered by the analyzed user (e.g., Figure 1 Then, the user search behavior data and the search query are input into a server that is deployed with a keyword intelligent resolution algorithm based on user behavior characteristics (for example, Figure 1 S as shown in the figure), wherein the server is able to use the keyword intelligent resolution algorithm based on user behavior characteristics to obtain multiple probability values ​​for the user search behavior data and the search query, and use the search term corresponding to the largest one of the multiple probability values ​​as the keyword.

[0044] After introducing the basic principles of the present application, various non-limiting embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0045] Figure 2 FIG. 1 is a flow chart of a keyword intelligent identification method based on user behavior characteristics according to an embodiment of the present application. Figure 2As shown, according to the embodiment of the present application, the keyword intelligent identification method based on user behavior characteristics includes the following steps: S110, obtaining the search behavior data of the analyzed user; S120, performing text preprocessing on the user search behavior data to obtain preprocessed user search behavior data; S130, performing word segmentation processing on the preprocessed user search behavior data and then passing it through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector; S140, obtaining a search query input by the analyzed user; S150, performing word segmentation processing on the search query and then passing it through a text convolutional neural network model including a word embedding layer word embedding layer to obtain a sequence of search word embedding vectors; S160, passing the sequence of search word embedding vectors through a converter-based context encoder to obtain a sequence of context search word feature vectors; S170, calculating the semantic association matrix between each context search word feature vector in the sequence of context search word feature vectors and the user search behavior semantic understanding feature vector to obtain multiple semantic association matrices; S180, passing the multiple semantic association matrices through classifiers respectively to obtain multiple probability values; and, S190, taking the search word corresponding to the largest of the multiple probability values ​​as a keyword.

[0046] Figure 3 FIG. 1 is a schematic diagram of the architecture of a keyword intelligent identification method based on user behavior characteristics according to an embodiment of the present application. Figure 3 As shown, in the network architecture, first, the search behavior data of the analyzed user is obtained; then, the user search behavior data is subjected to text preprocessing to obtain the preprocessed user search behavior data; then, the preprocessed user search behavior data is subjected to word segmentation processing and then passed through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector; then, the search query input by the analyzed user is obtained; then, the search query is subjected to word segmentation processing and then passed through a word embedding layer to obtain a sequence of search word embedding vectors; then, the sequence of search word embedding vectors is passed through a converter-based context encoder to obtain a sequence of context search word feature vectors; then, the semantic association matrix between each context search word feature vector in the sequence of context search word feature vectors and the user search behavior semantic understanding feature vector is calculated to obtain multiple semantic association matrices; then, the multiple semantic association matrices are respectively passed through a classifier to obtain multiple probability values; finally, the search word corresponding to the largest of the multiple probability values ​​is used as a keyword.

[0047] More specifically, in step S110, the search behavior data of the analyzed user is obtained. The user's search and browsing behavior can often reflect their real needs and interests, without being disturbed by text information such as titles and summaries. Compared with the traditional keyword recognition method based on text content, the method based on user behavior can more accurately reflect the user's interests and behavioral habits. In addition, the method based on user behavior can also characterize and analyze users in a more fine-grained manner.

[0048] More specifically, in step S120, the user search behavior data is subjected to text preprocessing to obtain preprocessed user search behavior data. User search behavior data usually contains a lot of useless or repeated information, which has no practical effect on keyword identification and information understanding. In the technical solution of the present application, the user search behavior data is subjected to text preprocessing to obtain preprocessed user search behavior data.

[0049] More specifically, in step S130, the preprocessed user search behavior data is segmented and then passed through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector. Since the preprocessed user search behavior data is text data, in the technical solution of the present application, the preprocessed user search behavior data is segmented and then passed through a text convolutional neural network model including a word embedding layer to convert the text into a numerical representation, and extract semantic feature information from it, thereby obtaining a user search behavior semantic understanding feature vector. Here, the word embedding layer can map each word to a vector in a high-dimensional space, thereby retaining the semantic relationship between words; the text convolutional neural network model uses operations such as convolutional layers and pooling layers to extract features from the sequence of word embedding vectors, thereby obtaining a feature vector that can characterize the user's search behavior intention.

[0050] Accordingly, in a specific example, Figure 4 As shown, the preprocessed user search behavior data is segmented and then passed through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector, including: S131, performing word segmentation on the preprocessed user search behavior data to convert the preprocessed user search behavior data into a word sequence composed of multiple words; S132, using the word embedding layer of the text convolutional neural network model to map each word in the word sequence to a word vector to obtain a sequence of word embedding vectors; and, S133, passing the sequence of word embedding vectors through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector.

[0051] Accordingly, in a specific example, the sequence of word embedding vectors is passed through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector, including: using each layer of the text convolutional neural network model to perform convolution processing, mean pooling processing and non-linear activation processing on the input data in the forward pass of the layer so that the last layer of the text convolutional neural network model outputs the user search behavior semantic understanding feature vector, wherein the input of the first layer of the text convolutional neural network model is the sequence of word embedding vectors.

[0052] It should be understood that Convolutional Neural Network (CNN) is an artificial neural network that is widely used in image recognition and other fields. Convolutional neural network may include input layer, hidden layer and output layer, wherein the hidden layer may include convolution layer, pooling layer, activation layer and fully connected layer, etc. The previous layer performs corresponding operations according to the input data and outputs the operation results to the next layer. The input initial data is processed through multiple layers of operations to obtain a final result.

[0053] More specifically, in step S140, a search query input by the analyzed user is obtained. In order to obtain the user's current search needs and convert them into numerical representations so as to compare and match them with their historical search behaviors, in the technical solution of the present application, first, the search query input by the analyzed user is obtained; then, the search query is segmented and passed through a word embedding layer to obtain a sequence of search word embedding vectors.

[0054] More specifically, in step S150, the search query is segmented and then passed through a word embedding layer to obtain a sequence of search word embedding vectors.

[0055] More specifically, in step S160, the sequence of search term embedding vectors is passed through a converter-based context encoder to obtain a sequence of context search term feature vectors. Considering the existence of contextual relationships between search terms, if such contextual relationships can be captured, the effect of subsequent classification can be greatly improved. In the technical solution of the present application, the sequence of search term embedding vectors is passed through a converter-based context encoder to obtain a sequence of context search term feature vectors. Here, the converter-based context encoder uses a self-attention mechanism to capture the correlation between each element in the sequence and the global semantic information of each element as a background.

[0056] It should be understood that through the context encoder, the relationship between a certain word segmentation and other word segmentations in the vector representation sequence can be analyzed to obtain the corresponding feature information. The context encoder aims to mine the hidden patterns between the contexts in the word sequence. Optionally, the encoder includes: CNN (Convolutional Neural Network), Recursive NN (Recursive Neural Network), Language Model, etc. The CNN-based method has a better extraction effect on local features, but it is not effective for the long-term dependency problem in sentences. Therefore, the encoder based on Bi-LSTM (Long Short-Term Memory) is widely used. Recursive NN treats sentences as tree structures rather than sequences. In theory, it has stronger representation capabilities, but it has weaknesses such as difficulty in sample annotation, easy gradient disappearance in deep layers, and difficulty in parallel calculation. Therefore, it is rarely used in practical applications. Transformer is a widely used network structure that has the characteristics of both CNN and RNN. It has a good extraction effect on global features and has certain advantages in parallel computing compared to RNN (recurrent neural network).

[0057] Accordingly, in a specific example, Figure 5 As shown, the sequence of search term embedding vectors is passed through a converter-based context encoder to obtain a sequence of context search term feature vectors, including: S161, one-dimensionally arranging the sequence of search term embedding vectors to obtain a global word sequence feature vector; S162, calculating the product between the global word sequence feature vector and the transposed vector of each search term embedding vector in the sequence of search term embedding vectors to obtain a plurality of self-attention association matrices; S163, respectively normalizing each of the plurality of self-attention association matrices to obtain a plurality of standardized self-attention association matrices; S164, passing each of the plurality of standardized self-attention association matrices through a Softmax classification function to obtain a plurality of probability values; and, S165, respectively weighting each of the search term embedding vectors in the sequence of search term embedding vectors using each of the plurality of probability values ​​as a weight to obtain the sequence of context search term feature vectors.

[0058] More specifically, in step S170, a semantic association matrix between each context search term feature vector in the sequence of context search term feature vectors and the user search behavior semantic understanding feature vector is calculated to obtain multiple semantic association matrices. In this way, the semantic similarity between the search term and the user search behavior can be measured, thereby obtaining multiple semantic association matrices.

[0059] Accordingly, in a specific example, calculating the semantic association matrix between each context retrieval word feature vector in the sequence of the context retrieval word feature vector and the user search behavior semantic understanding feature vector to obtain multiple semantic association matrices includes: calculating the semantic association matrix between each context retrieval word feature vector in the sequence of the context retrieval word feature vector and the user search behavior semantic understanding feature vector using the following association formula to obtain the multiple semantic association matrices; wherein the association formula is: ,in, represents each context retrieval word feature vector in the sequence of context retrieval word feature vectors, represents the transposed vector of a vector, represents the semantic understanding feature vector of the user search behavior, represents each semantic association matrix in the plurality of semantic association matrices, Represents vector multiplication.

[0060] More specifically, in step S180, the multiple semantic association matrices are respectively passed through a classifier to obtain multiple probability values. The classifier is a neural network model, whose input is a semantic association matrix, and whose output is a probability value, indicating the possibility of whether the search term is a keyword. In the technical solution of the present application, by selecting the search term corresponding to the one with the largest probability value as the keyword, the keyword that best represents the user's needs and interests can be selected.

[0061] More specifically, in step S190, the search term corresponding to the largest one of the multiple probability values ​​is used as the keyword.

[0062] Accordingly, in a specific example, a training step is also included: training the text convolutional neural network model including the word embedding layer, the converter-based context encoder and the classifier; wherein, Figure 6As shown, the training step includes: S201, obtaining the training search behavior data of the analyzed user and the real value of the keyword; S202, performing text preprocessing on the training search behavior data to obtain the user search behavior data after training preprocessing; S203, performing word segmentation processing on the user search behavior data after training preprocessing and passing it through the text convolutional neural network model including the word embedding layer to obtain the training user search behavior semantic understanding feature vector; S204, obtaining the training retrieval query input by the analyzed user; S205, performing word segmentation processing on the training retrieval query and passing it through the word embedding layer to obtain a sequence of training retrieval word embedding vectors; S206, passing the sequence of training retrieval word embedding vectors through the converter-based context encoder to obtain a sequence of training context retrieval word feature vectors; S207, calculate the semantic association matrix between each training context retrieval word feature vector in the sequence of the training context retrieval word feature vectors and the training user search behavior semantic understanding feature vector to obtain multiple training semantic association matrices; S208, calculate the streaming refinement loss function value of each training context retrieval word feature vector and the training user search behavior semantic understanding feature vector; S209, pass the multiple training semantic association matrices through the classifier respectively to obtain multiple classification loss function values; and, S210, based on the weighted sum of the streaming refinement loss function value and the multiple classification loss function values ​​as the loss function, and train the text convolutional neural network model including the word embedding layer, the converter-based context encoder and the classifier through the direction propagation of gradient descent.

[0063] Here, when calculating the semantic association matrix between each training context retrieval term feature vector in the sequence of the training context retrieval term feature vector and the training user search behavior semantic understanding feature vector to obtain multiple training semantic association matrices, it is essentially mapping the serialized context encoding features of the search term represented by each training context retrieval term feature vector and the textualized sequence semantic feature representation of the user search behavior represented by the training user search behavior semantic understanding feature vector to a high-dimensional association feature space. Therefore, if the correlation between the training context retrieval term feature vector and the training user search behavior semantic understanding feature vector in the association space dimension between the serialized text semantic feature representation and the high-dimensional feature can be improved, the expression effect of the training semantic association matrix can be improved. Based on this, the applicant of the present application further introduces a streaming refinement loss function for each of the training context retrieval term feature vectors and the training user search behavior semantic understanding feature vector in addition to the classification loss function based on the training semantic association matrix.

[0064] Accordingly, in a specific example, calculating the streaming refinement loss function values ​​of the respective training context retrieval word feature vectors and the training user search behavior semantic understanding feature vectors includes: calculating the streaming refinement loss function values ​​of the respective training context retrieval word feature vectors and the training user search behavior semantic understanding feature vectors using the following loss formula; ,in, represents the feature vectors of each training context retrieval word, represents the semantic understanding feature vector of the training user search behavior, represents the square of the vector's two-norm, and and represent position-wise subtraction and multiplication of vectors, respectively. represents the exponential operation of a vector, wherein the exponential operation of a vector represents the calculation of the value of a natural exponential function raised to the power of the eigenvalues ​​at each position in the vector, Represents the streaming refinement loss function value.

[0065] Here, the streaming refinement loss function is based on the transformation of the serialized encoding streaming distribution of each training context retrieval term feature vector and the training user search behavior semantic understanding feature vector in the text to the spatial distribution in the high-dimensional associated feature space, and realizes the super-resolution improvement of the spatial distribution in the high-dimensional associated feature space by synchronously performing interpolation under the sequence distribution of the vector, thereby providing a more refined alignment of the distribution differences in the high-dimensional associated feature space by balancing the mutual probability relationship under the sequence, so as to jointly present the cross-dimensional (inter-dimensional) contextual associations in the serialized text semantic feature dimension and the associated feature space dimension, thereby improving the expression effect of the multiple training semantic association matrices, and then improving the accuracy of the multiple probability values ​​obtained by the classifier of the multiple training semantic association matrices.

[0066] In summary, the method for intelligently distinguishing keywords based on user behavior characteristics according to the embodiment of the present application first performs text preprocessing on the user search behavior data and performs word segmentation processing, and then passes through a text convolutional neural network model to obtain a user search behavior semantic understanding feature vector. Next, the search query input by the analyzed user is subjected to word segmentation processing and then passes through a word embedding layer to obtain a sequence of search word embedding vectors. Then, the sequence of search word embedding vectors passes through a context encoder to obtain a sequence of context search word feature vectors. Next, the semantic association matrix between each context search word feature vector and the user search behavior semantic understanding feature vector is calculated to obtain multiple semantic association matrices. Finally, the multiple semantic association matrices are respectively passed through a classifier to obtain multiple probability values ​​and the search word corresponding to the largest of the multiple probability values ​​is used as a keyword. In this way, keywords can be intelligently distinguished.

[0067] Figure 7 FIG. 1 is a block diagram of a keyword intelligent identification system 100 based on user behavior characteristics according to an embodiment of the present application. Figure 7 As shown, according to the embodiment of the present application, the keyword intelligent resolution system 100 based on user behavior characteristics includes: a search behavior data acquisition module 110, which is used to obtain the search behavior data of the analyzed user; a text preprocessing module 120, which is used to perform text preprocessing on the user search behavior data to obtain the preprocessed user search behavior data; an embedding convolutional coding module 130, which is used to perform word segmentation processing on the preprocessed user search behavior data and then pass it through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector; a user input data acquisition module 140, which is used to obtain the search query input by the analyzed user; an embedding module 150, which is used to perform word segmentation processing on the preprocessed user search behavior data and then pass it through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector; After the word segmentation process, the word embedding layer is used to obtain a sequence of search word embedding vectors; a context encoding module 160 is used to pass the sequence of search word embedding vectors through a converter-based context encoder to obtain a sequence of context search word feature vectors; a semantic association module 170 is used to calculate the semantic association matrix between each context search word feature vector in the sequence of context search word feature vectors and the user search behavior semantic understanding feature vector to obtain multiple semantic association matrices; a classification module 180 is used to pass the multiple semantic association matrices through a classifier respectively to obtain multiple probability values; and a keyword acquisition module 190 is used to use the search word corresponding to the largest of the multiple probability values ​​as a keyword.

[0068] In one example, in the above-mentioned keyword intelligent resolution system 100 based on user behavior characteristics, the embedded convolutional coding module 130 is used to: perform word segmentation on the preprocessed user search behavior data to convert the preprocessed user search behavior data into a word sequence composed of multiple words; use the word embedding layer of the text convolutional neural network model to map each word in the word sequence to a word vector to obtain a sequence of word embedding vectors; and, pass the sequence of word embedding vectors through the text convolutional neural network model to obtain a semantic understanding feature vector of the user search behavior.

[0069] In one example, in the above-mentioned keyword intelligent resolution system 100 based on user behavior characteristics, the sequence of word embedding vectors is passed through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector, including: using each layer of the text convolutional neural network model to perform convolution processing, mean pooling processing and non-linear activation processing on the input data in the forward pass of the layer so that the last layer of the text convolutional neural network model outputs the user search behavior semantic understanding feature vector, wherein the input of the first layer of the text convolutional neural network model is the sequence of word embedding vectors.

[0070] In one example, in the above-mentioned keyword intelligent resolution system 100 based on user behavior characteristics, the context encoding module 160 is used to: arrange the sequence of search term embedding vectors in one dimension to obtain a global word sequence feature vector; calculate the product between the global word sequence feature vector and the transposed vector of each search term embedding vector in the sequence of search term embedding vectors to obtain multiple self-attention association matrices; standardize each of the multiple self-attention association matrices to obtain multiple standardized self-attention association matrices; pass each of the multiple standardized self-attention association matrices through the Softmax classification function to obtain multiple probability values; and, weight each search term embedding vector in the sequence of search term embedding vectors using each of the multiple probability values ​​as a weight to obtain the sequence of context search term feature vectors.

[0071] In one example, in the above-mentioned keyword intelligent resolution system 100 based on user behavior characteristics, the semantic association module 170 is used to: calculate the semantic association matrix between each context search word feature vector in the sequence of the context search word feature vector and the user search behavior semantic understanding feature vector using the following association formula to obtain the multiple semantic association matrices; wherein the association formula is: ,in, represents each context retrieval word feature vector in the sequence of context retrieval word feature vectors, represents the transposed vector of a vector, represents the semantic understanding feature vector of the user search behavior, represents each semantic association matrix in the plurality of semantic association matrices, Represents vector multiplication.

[0072] In one example, in the above-mentioned keyword intelligent resolution system 100 based on user behavior characteristics, it also includes a training module for training the text convolutional neural network model including the word embedding layer, the converter-based context encoder and the classifier; wherein the training module is used to: obtain the training search behavior data of the analyzed user, and the true value of the keyword; perform text preprocessing on the training search behavior data to obtain the user search behavior data after training preprocessing; perform word segmentation processing on the user search behavior data after training preprocessing and pass it through the text convolutional neural network model including the word embedding layer to obtain the training user search behavior semantic understanding feature vector; obtain the training retrieval query input by the analyzed user; perform word segmentation processing on the training retrieval query and pass it through the word embedding layer to obtain a sequence of training retrieval word embedding vectors; embed the training retrieval word into A sequence of quantities is passed through the converter-based context encoder to obtain a sequence of training context retrieval word feature vectors; a semantic association matrix between each training context retrieval word feature vector in the sequence of training context retrieval word feature vectors and the training user search behavior semantic understanding feature vector is calculated to obtain multiple training semantic association matrices; a streaming refinement loss function value of each training context retrieval word feature vector and the training user search behavior semantic understanding feature vector is calculated; the multiple training semantic association matrices are respectively passed through the classifier to obtain multiple classification loss function values; and, based on the weighted sum of the streaming refinement loss function value and the multiple classification loss function values ​​as the loss function, the text convolutional neural network model including the word embedding layer, the converter-based context encoder and the classifier are trained by directional propagation of gradient descent.

[0073] In one example, in the above-mentioned keyword intelligent resolution system 100 based on user behavior characteristics, the streaming refinement loss function values ​​of the respective training context retrieval word feature vectors and the training user search behavior semantic understanding feature vectors are calculated, including: calculating the streaming refinement loss function values ​​of the respective training context retrieval word feature vectors and the training user search behavior semantic understanding feature vectors using the following loss formula; ,in, represents the feature vectors of each training context retrieval word, represents the semantic understanding feature vector of the training user search behavior, represents the square of the vector's two-norm, and and represent position-wise subtraction and multiplication of vectors, respectively. represents the exponential operation of a vector, wherein the exponential operation of a vector represents the calculation of the value of a natural exponential function raised to the power of the eigenvalues ​​at each position in the vector, Represents the streaming refinement loss function value.

[0074] Here, those skilled in the art can understand that the specific functions and operations of each module in the above-mentioned keyword intelligent resolution system based on user behavior characteristics 100 have been referred to above. Figures 1 to 6 It has been introduced in detail in the description of the keyword intelligent resolution method based on user behavior characteristics, and therefore, its repeated description will be omitted.

[0075] As described above, the keyword intelligent resolution system 100 based on user behavior characteristics according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a keyword intelligent resolution algorithm based on user behavior characteristics. In one example, the keyword intelligent resolution system 100 based on user behavior characteristics according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the keyword intelligent resolution system 100 based on user behavior characteristics can be a software module in the operating system of the wireless terminal, or can be an application developed for the wireless terminal; of course, the keyword intelligent resolution system 100 based on user behavior characteristics can also be one of the many hardware modules of the wireless terminal.

[0076] Alternatively, in another example, the keyword intelligent resolution system based on user behavior characteristics 100 and the wireless terminal may also be separate devices, and the keyword intelligent resolution system based on user behavior characteristics 100 may be connected to the wireless terminal via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0077] According to another aspect of the present application, a non-volatile computer-readable storage medium is provided, on which computer-readable instructions are stored. When the instructions are executed by a computer, the above-mentioned method can be executed.

[0078] The program part of the technology can be considered as a "product" or "manufactured product" in the form of executable code and / or related data, which is participated in or realized by computer-readable media. Tangible and permanent storage media can include any memory or storage used by any computer, processor, or similar device or related module. For example, various semiconductor memories, tape drives, disk drives or any similar devices that can provide storage functions for software.

[0079] All or part of the software may sometimes communicate over a network, such as the Internet or other communications network. Such communications can load software from one computer device or processor to another. For example: from a server or host computer of a video target detection device to a hardware platform of a computer environment, or other computer environment that implements the system, or a system with similar functions related to providing information required for target detection. Therefore, another medium capable of transmitting software elements can also be used as a physical connection between local devices, such as light waves, radio waves, electromagnetic waves, etc., transmitted through cables, optical cables or air. Physical media used to carry carriers, such as cables, wireless connections or optical cables and the like, can also be considered as media that carry software. As used herein, unless limited to tangible "storage" media, other terms referring to computer or machine "readable media" refer to media involved in the process of the processor executing any instructions.

[0080] The present application uses specific words to describe the embodiments of the present application. For example, "first / second embodiment", "one embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or multiple times in different positions in this specification does not necessarily refer to the same embodiment. In addition, some features, structures or characteristics in one or more embodiments of the present application can be appropriately combined.

[0081] In addition, it will be appreciated by those skilled in the art that various aspects of the present application may be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0082] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined as such herein.

[0083] The above is an explanation of the present invention and should not be considered as a limitation thereof. Although several exemplary embodiments of the present invention have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present invention. Therefore, all such modifications are intended to be included within the scope of the present invention as defined in the claims. It should be understood that the above is an explanation of the present invention and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present invention is defined by the claims and their equivalents.

Claims

1. A keyword intelligent identification method based on user behavior characteristics, characterized in that: include: Obtain the search behavior data of the analyzed user; Performing text preprocessing on the user search behavior data to obtain preprocessed user search behavior data; After word segmentation processing is performed on the preprocessed user search behavior data, a text convolutional neural network model including a word embedding layer is used to obtain a user search behavior semantic understanding feature vector; Obtaining a search query input by the analyzed user; After word segmentation, the search query is passed through a word embedding layer to obtain a sequence of search word embedding vectors; Passing the sequence of search term embedding vectors through a transformer-based context encoder to obtain a sequence of context search term feature vectors; Calculating a semantic association matrix between each context retrieval word feature vector in the sequence of context retrieval word feature vectors and the user search behavior semantic understanding feature vector to obtain a plurality of semantic association matrices; Passing the plurality of semantic association matrices through classifiers respectively to obtain a plurality of probability values; as well as Using the search term corresponding to the largest of the multiple probability values ​​as the keyword; The method further includes a training step of: training the text convolutional neural network model including the word embedding layer, the converter-based context encoder, and the classifier; Wherein, the training step comprises: Obtain the training search behavior data of the analyzed user and the real value of the keyword; Performing text preprocessing on the training search behavior data to obtain training preprocessed user search behavior data; After word segmentation processing is performed on the user search behavior data after the training preprocessing, the data is passed through the text convolutional neural network model including the word embedding layer to obtain a semantic understanding feature vector of the training user search behavior; Obtaining a training search query input by the analyzed user; The training search query is processed by word segmentation and then passed through a word embedding layer to obtain a sequence of training search word embedding vectors; Passing the sequence of training search term embedding vectors through the transformer-based context encoder to obtain a sequence of training context search term feature vectors; Calculating a semantic association matrix between each training context retrieval word feature vector in the sequence of the training context retrieval word feature vector and the training user search behavior semantic understanding feature vector to obtain a plurality of training semantic association matrices; Calculating the streaming refinement loss function values ​​of each training context search word feature vector and the training user search behavior semantic understanding feature vector; Passing the plurality of training semantic association matrices through the classifier respectively to obtain a plurality of classification loss function values; and Based on the weighted sum of the streaming refinement loss function value and the multiple classification loss function values ​​as a loss function, the text convolutional neural network model including the word embedding layer, the converter-based context encoder and the classifier are trained by directional propagation of gradient descent; The calculation of the streaming refinement loss function values ​​of the training context search word feature vectors and the training user search behavior semantic understanding feature vectors includes: The streaming refinement loss function value of each training context search word feature vector and the training user search behavior semantic understanding feature vector is calculated using the following loss formula; ;in, represents the feature vectors of each training context retrieval word, represents the semantic understanding feature vector of the training user search behavior, represents the square of the vector's two-norm, and and represent position-wise subtraction and multiplication of vectors, respectively. represents the exponential operation of a vector, wherein the exponential operation of a vector represents the calculation of the value of a natural exponential function raised to the power of the eigenvalues ​​at each position in the vector, Represents the streaming refinement loss function value.

2. The keyword intelligent identification method based on user behavior characteristics according to claim 1 is characterized in that: The pre-processed user search behavior data is segmented and then passed through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector, including: Performing word segmentation processing on the pre-processed user search behavior data to convert the pre-processed user search behavior data into a word sequence consisting of multiple words; Using the word embedding layer of the text convolutional neural network model, mapping each word in the word sequence to a word vector to obtain a sequence of word embedding vectors; and The sequence of word embedding vectors is passed through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector.

3. The keyword intelligent identification method based on user behavior characteristics according to claim 2 is characterized in that: Passing the sequence of word embedding vectors through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector, including: Use each layer of the text convolutional neural network model to perform convolution processing, mean pooling processing and nonlinear activation processing on the input data in the forward pass of the layer so that the last layer of the text convolutional neural network model outputs the user search behavior semantic understanding feature vector, wherein the input of the first layer of the text convolutional neural network model is a sequence of word embedding vectors.

4. The keyword intelligent identification method based on user behavior characteristics according to claim 3 is characterized in that: Passing the sequence of search term embedding vectors through a transformer-based context encoder to obtain a sequence of context search term feature vectors, including: Arranging the sequence of the search word embedding vectors in one dimension to obtain a global word sequence feature vector; Calculating the product of the global word sequence feature vector and the transposed vector of each search word embedding vector in the sequence of search word embedding vectors to obtain a plurality of self-attention association matrices; Normalizing each of the plurality of self-attention association matrices to obtain a plurality of standardized self-attention association matrices; Passing each of the plurality of standardized self-attention association matrices through a Softmax classification function to obtain a plurality of probability values; and Each of the plurality of probability values ​​is used as a weight to weight each of the search term embedding vectors in the sequence of search term embedding vectors to obtain the sequence of context search term feature vectors.

5. The keyword intelligent identification method based on user behavior characteristics according to claim 4 is characterized in that: Calculating the semantic association matrix between each context retrieval word feature vector in the sequence of the context retrieval word feature vector and the user search behavior semantic understanding feature vector to obtain multiple semantic association matrices, including: Calculating the semantic association matrix between each context retrieval word feature vector in the sequence of the context retrieval word feature vector and the user search behavior semantic understanding feature vector using the following association formula to obtain the multiple semantic association matrices; Wherein, the association formula is: ;in, represents each context retrieval word feature vector in the sequence of context retrieval word feature vectors, represents the transposed vector of a vector, represents the semantic understanding feature vector of the user search behavior, represents each semantic association matrix in the plurality of semantic association matrices, Represents vector multiplication.

6. A keyword intelligent identification system based on user behavior characteristics, characterized in that: include: A search behavior data acquisition module, used to acquire the search behavior data of the analyzed user; A text preprocessing module, used for performing text preprocessing on the user search behavior data to obtain preprocessed user search behavior data; An embedding convolutional coding module, used to perform word segmentation processing on the pre-processed user search behavior data and then pass it through a text convolutional neural network model including a word embedding layer to obtain a user search behavior semantic understanding feature vector; A user input data acquisition module, used to acquire a search query input by the analyzed user; An embedding module, configured to perform word segmentation processing on the search query and then pass it through a word embedding layer to obtain a sequence of search word embedding vectors; A context encoding module, used for passing the sequence of search term embedding vectors through a converter-based context encoder to obtain a sequence of context search term feature vectors; A semantic association module, used for calculating a semantic association matrix between each context search word feature vector in the sequence of the context search word feature vector and the user search behavior semantic understanding feature vector to obtain a plurality of semantic association matrices; A classification module, used for passing the plurality of semantic association matrices through a classifier respectively to obtain a plurality of probability values; as well as A keyword acquisition module, used to use the search term corresponding to the largest one among the multiple probability values ​​as the keyword; It also includes a training module for training the text convolutional neural network model including the word embedding layer, the converter-based context encoder and the classifier; wherein the training module is used to: obtain the training search behavior data of the analyzed user and the true value of the keyword; perform text preprocessing on the training search behavior data to obtain the pre-processed user search behavior data; perform word segmentation on the pre-processed user search behavior data and pass it through the text convolutional neural network model including the word embedding layer to obtain the training user search behavior semantic understanding feature vector; obtain the training search query input by the analyzed user; perform word segmentation on the training search query and pass it through the word embedding layer to obtain a sequence of training search word embedding vectors; pass the sequence of training search word embedding vectors through the converter-based context encoder. The following text encoder is used to obtain a sequence of training context retrieval word feature vectors; the semantic association matrix between each training context retrieval word feature vector in the sequence of training context retrieval word feature vectors and the training user search behavior semantic understanding feature vector is calculated to obtain multiple training semantic association matrices; the streaming refinement loss function value of each training context retrieval word feature vector and the training user search behavior semantic understanding feature vector is calculated; the multiple training semantic association matrices are respectively passed through the classifier to obtain multiple classification loss function values; and, based on the weighted sum of the streaming refinement loss function value and the multiple classification loss function values ​​as the loss function, the text convolutional neural network model including the word embedding layer, the converter-based context encoder and the classifier are trained by the direction propagation of gradient descent; Wherein, calculating the streaming refinement loss function value of each training context retrieval word feature vector and the training user search behavior semantic understanding feature vector includes: calculating the streaming refinement loss function value of each training context retrieval word feature vector and the training user search behavior semantic understanding feature vector using the following loss formula; ;in, represents the feature vectors of each training context retrieval word, represents the semantic understanding feature vector of the training user search behavior, represents the square of the vector's two-norm, and and represent position-wise subtraction and multiplication of vectors, respectively. represents the exponential operation of a vector, wherein the exponential operation of a vector represents the calculation of the value of a natural exponential function raised to the power of the eigenvalues ​​at each position in the vector, Represents the streaming refinement loss function value.

7. The keyword intelligent identification system based on user behavior characteristics according to claim 6 is characterized in that: The embedded convolutional coding module is used to: Performing word segmentation processing on the pre-processed user search behavior data to convert the pre-processed user search behavior data into a word sequence consisting of multiple words; Using the word embedding layer of the text convolutional neural network model, mapping each word in the word sequence to a word vector to obtain a sequence of word embedding vectors; and The sequence of word embedding vectors is passed through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector.

8. The keyword intelligent identification system based on user behavior characteristics according to claim 7 is characterized in that: Passing the sequence of word embedding vectors through the text convolutional neural network model to obtain the user search behavior semantic understanding feature vector, including: Use each layer of the text convolutional neural network model to perform convolution processing, mean pooling processing and nonlinear activation processing on the input data in the forward pass of the layer so that the last layer of the text convolutional neural network model outputs the user search behavior semantic understanding feature vector, wherein the input of the first layer of the text convolutional neural network model is a sequence of word embedding vectors.

Images