Search term category identification method and model training method and device thereof

Abstract

The application discloses a search word category identification method and a training method and device of a search word category identification model, and the training method of the search word category identification model comprises the following steps: acquiring a plurality of training samples; constructing a pre-training model; inputting a search word segmentation sequence into an embedding layer to obtain a token vector, a segment vector and a mask vector; inputting the vectors into an encoder layer for feature extraction to obtain text features, first word features and character features; performing aggregation processing on the character features to obtain second word features; taking the text features, the first word features and the second word features as inputs of a full connection layer, taking real category target labels as outputs, training network parameters of the pre-training model, and obtaining a search word category identification model. Through the search word category identification model, the belonging probability of a word containing a plurality of categories is judged in combination with context, the ambiguity problem in category identification is greatly improved, and the accuracy of category identification is higher.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular to a method for identifying search term categories and a training method and apparatus for its model. Background Technology

[0002] Search term category identification involves assigning categories to search terms after they have been segmented and providing corresponding probability values. By analyzing the search terms entered by users, it is helpful to analyze users' core search needs, thereby improving the user experience.

[0003] Currently, mainstream category recognition methods are mainly based on accumulated category databases and exact matching. The more accurate the category database, the more categories it accumulates, leading to a higher probability of noise in database queries, thus increasing the labor intensity of manual maintenance and querying. Moreover, since each word may have multiple categories, meaning that a word belongs to different categories in different scenarios, it is difficult to resolve ambiguities in category recognition based solely on the database and the word itself, resulting in lower accuracy in category recognition. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of low accuracy in category identification of multiple categories of words in the prior art, and to provide a search term category identification method and a training method and apparatus for its model.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution:

[0006] According to a first aspect of the present invention, a method for training a search term category recognition model is provided, the method comprising:

[0007] Obtain multiple training samples; wherein each training sample includes a search word segmentation sequence corresponding to the search text and a real class target tag corresponding to each search word in the search word segmentation sequence;

[0008] Construct a pre-trained model, which includes an embedding layer, an encoder layer, and a fully connected layer;

[0009] The search word segmentation sequence is input into the embedding layer to obtain a token vector, a segment vector, and a mask vector; wherein, the embedding layer includes a token embedding layer, a segment embedding layer, and a mask layer;

[0010] The token vector, the mask vector, and the segment vector are input into the encoder layer for feature extraction to obtain the text features corresponding to the search word segmentation sequence, the first word features corresponding to the search word segmentation, and the character features corresponding to the characters of the search word segmentation; wherein, the encoder layer is a multi-layer Transformer structure;

[0011] Based on the segment vector, the character features are aggregated to obtain the second word features of the search word segmentation;

[0012] Using the text features, the first word features, and the second word features as inputs to the fully connected layer, and the real class target tags as outputs, the network parameters of the pre-trained model are trained to obtain the search term category recognition model.

[0013] Preferably, the step of inputting the search word segmentation sequence into the embedding layer to obtain the token vector, segment vector, and mask vector includes:

[0014] The token embedding layer inserts a CLS token at the beginning of the search word segmentation sequence and a SEP token at the end of each search word, mapping the CLS token, the SEP token, and the characters of the search word segmentation sequence to a token vector.

[0015] The segment embedding layer encodes and maps the position of the search word in the search word sequence into a segment vector.

[0016] The characters in the search word segmentation sequence are replaced sequentially using the MASK marker through the mask layer, and the replaced search word segmentation sequence is mapped to a mask vector.

[0017] Preferably, the step of aggregating the character features based on the segment vector includes:

[0018] Based on the segment vector, obtain the character fragments corresponding to each search word;

[0019] The character features under the character fragment are aggregated to obtain the second word features corresponding to the search word segment.

[0020] Preferably, the step of acquiring multiple training samples is further characterized by:

[0021] Retrieve multiple search texts from the target industry;

[0022] For each search text, the search text is preprocessed to obtain the training samples.

[0023] Preferably, the step of preprocessing the search text includes:

[0024] The search text is cleaned; the data cleaning includes filtering out meaningless symbols and removing spaces.

[0025] The search text after data cleaning is matched with a thesaurus related to the target industry to obtain multiple search terms;

[0026] Multiple search word segments are generated based on the search word segmentation connections;

[0027] The optimal search word segmentation sequence is selected from the search word segmentation sequence as the training sample.

[0028] According to a second aspect of the present invention, a method for identifying search term categories is provided, the steps of which include:

[0029] Obtain the search text to be recognized;

[0030] The search text to be identified is preprocessed to obtain a search word segmentation sequence to be identified, which includes multiple search words to be identified.

[0031] The search word segmentation sequence to be identified is input into the search word category recognition model to obtain at least one category result corresponding to each search word to be identified.

[0032] The search term category recognition model is obtained through the training method of the search term category recognition model of the present invention.

[0033] According to a third aspect of the present invention, a training apparatus for a search term category recognition model is provided, wherein the fully connected layer of the search term category recognition model includes multiple parallel fully connected layers, and the training apparatus for the search term category recognition model includes:

[0034] The first acquisition module is used to acquire multiple training samples; wherein, the training samples include a search word segmentation sequence corresponding to the search text and a real class target tag corresponding to each search word in the search word segmentation sequence;

[0035] The model building module is used to build a pre-trained model, which includes an embedding layer, an encoder layer, and a fully connected layer.

[0036] The input conversion module is used to input the search word segmentation sequence into the embedding layer to obtain a token vector, a segment vector, and a mask vector; wherein, the embedding layer includes a token embedding layer, a segment embedding layer, and a mask layer;

[0037] The feature extraction module is used to input the token vector, the mask vector, and the segment vector into the encoder layer for feature extraction, to obtain the text features corresponding to the search word segmentation sequence, the first word features corresponding to the search word segmentation, and the character features corresponding to the characters of the search word segmentation; wherein, the encoder layer is a multi-layer Transformer structure;

[0038] The word conversion module is used to aggregate the word features based on the segment vector to obtain the second word features of the search word segmentation;

[0039] The training module is used to train the network parameters of the pre-trained model with the text features, the first word features, and the second word features as inputs to the fully connected layer and the real class tags as outputs, so as to obtain the search term category recognition model.

[0040] According to a fourth aspect of the present invention, a search term category identification device is provided, the search term category identification device comprising:

[0041] The second acquisition module is used to acquire the search text to be recognized;

[0042] The preprocessing module is used to preprocess the search text to be identified to obtain a search word segmentation sequence to be identified, wherein the search word segmentation sequence to be identified includes multiple search words to be identified;

[0043] The category recognition module is used to input the search word segmentation sequence to be recognized into the search word category recognition model to obtain at least one category result corresponding to each search word to be recognized.

[0044] The search term category recognition model is obtained through the training device of the search term category recognition model of the present invention.

[0045] According to a fifth aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a training method for a search term category recognition model of the present invention or a search term category recognition method of the present invention.

[0046] According to a sixth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a training method for a search term category recognition model of the present invention or a search term category recognition method of the present invention.

[0047] The positive and progressive effects of this invention are as follows:

[0048] This invention performs category identification for search terms based on the entire search text. It uses a search term category identification model to obtain semantic information from the search text, the semantic information of each search term, and the semantic information of each character. For each search term, character features encompassing contextual features are extracted, and word features are obtained by aggregating these character features. Then, based on the contextual and word features of each search term, the corresponding category and its probability are derived. This invention, by combining a search term category identification model with contextual and word features, determines the probability of category affiliation for words containing multiple categories, significantly improving the ambiguity problem in category identification and achieving higher accuracy. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the training method of the search term category recognition model in Embodiment 1 of the present invention.

[0050] Figure 2 This is a schematic diagram of the training method for the search term category recognition model in Embodiment 1 of the present invention.

[0051] Figure 3 This is a flowchart illustrating the search term category identification method of Embodiment 2 of the present invention.

[0052] Figure 4 This is a schematic diagram of the training device for the search term category recognition model in Embodiment 3 of the present invention.

[0053] Figure 5 This is a schematic diagram of the search term category identification device according to Embodiment 4 of the present invention.

[0054] Figure 6 This is a schematic diagram of the electronic device according to Embodiment 5 of the present invention. Detailed Implementation

[0055] The present invention will be further illustrated by way of embodiments below, but the present invention is not limited to the scope of the embodiments described herein.

[0056] Example 1

[0057] This embodiment provides a training method for a search term category recognition model, such as... Figure 1 As shown, the training method for this search term category recognition model includes the following steps:

[0058] S11. Obtain multiple training samples; wherein, each training sample includes a search word segmentation sequence corresponding to the search text and a real class target tag corresponding to each search word in the search word segmentation sequence.

[0059] In this embodiment, the data source for the training samples is primarily search text, requiring the acquisition of multiple search texts from the target industry. For ease of explanation, the target industry is defined as tourism-related, although this embodiment is not limited to tourism scenarios. As an optional implementation, the search text entered by the user in the front-end search box is obtained from a tourism-related search engine, and then preprocessed, including data cleaning, Chinese word segmentation, and constructing a search word segmentation sequence.

[0060] See Figure 2 The system cleanses the user-input search text to obtain unsegmented search text. Data cleaning includes removing irrelevant symbols, converting traditional Chinese characters to simplified Chinese, deleting extra spaces, and applying length limits. For example, if the input search text is "Lijiang-Shangri-La", the cleaned text will be "Lijiang Shangri-La".

[0061] As an optional implementation, the unsegmented search text is matched with a thesaurus related to the target industry to obtain corresponding search terms. For example, segmenting "Lijiang Shangri-La" yields search terms including "Lijiang", "Shangri-La", and "Shangri". Then, multiple search term sequences are generated based on these search term connections. For instance, the search term sequences obtained from "Lijiang", "Shangri-La", and "Shangri" include "Lijiang Shangri-La" and "Lijiang Shangri-La". Finally, the optimal search term sequence is selected as the training sample.

[0062] As one optional implementation, when selecting the optimal search word segmentation sequence, the path span of the search word segmentation sequences is compared. Typically, the search word segmentation sequence with the larger path span is selected as the optimal search word segmentation sequence, where the path refers to the text length of each search word. As another optional implementation, when the path span is consistent, the word length of each search word can be obtained. The search word segmentation sequence with the longer word length and a relatively later position in its sequence can be selected as the optimal search word segmentation sequence, or the search word segmentation sequence with more even word lengths can be selected as the optimal search word segmentation sequence. For example, assuming "Lijiang Shangri-La" is segmented into "Lijiang Shangri-La" and "Lijiang Shangri-La", where "Shangri-La" is later in the sequence, "Lijiang Shangri-La" is selected as the optimal search word segmentation sequence, and "Lijiang Shangri-La" is used as the training sample.

[0063] S12. Construct a pre-trained model, which includes an embedding layer, an encoder layer, and a fully connected layer.

[0064] As an optional implementation, the pre-training module of this embodiment is an improved BERT (Bidirectional Encoder Representation Based on Transformer, a pre-trained language representation model) model. BERT is pre-trained using MLM (Mask Language Model) and employs a deep bidirectional Transformer (encoder) component to generate deep bidirectional language representations, so that the output of the BERT model covers contextual semantic information.

[0065] S13. Input the search word segmentation sequence into the embedding layer to obtain the token vector, segment vector and mask vector; wherein, the embedding layer includes the token embedding layer, the segment embedding layer and the mask layer.

[0066] The embedding layer is used to convert the input search word segmentation sequence into a vector representation. The search word segmentation sequence is input into the token embedding layer, the segment embedding layer and the mask layer respectively to obtain the token vector, segment vector and mask vector.

[0067] In this embodiment, when inputting the search word segmentation sequence into the token embedding layer, CLS and SEP tokens also need to be input. As an optional implementation, [CLS] is inserted at the beginning of the search word segmentation sequence, and [SEP] is inserted at the end of each search word segment. After passing through the token embedding layer of the pre-trained model, the corresponding token vector is obtained, such as... Figure 2 As shown, the token vector corresponding to [CLS] is E. CLS The token vector corresponding to [SEP] is E. SEP The token vector corresponding to each character in the search word segmentation sequence is: This yields the token vector of the search word segmentation sequence. Where N is the number of characters in the search word segmentation sequence, i represents the i-th search word, and l i C represents the number of characters in the i-th search term. N This represents the Nth character.

[0068] In this embodiment, the segment vector represents the position of the search word in the search text. The search word sequence is input into the segment embedding layer, such as... Figure 2 As shown, the segment vector E of the search word segment sequence is obtained. segment ={E0,E0,…,E1,…,E1,…,E i ,…,E m E m}, where m represents the number of search terms.

[0069] In this embodiment, the mask vector represents randomly masking or replacing any word or phrase in the search text. During training, the smallest input unit token in the entire sentence is randomly selected for masking or replacement. Then, a pre-trained model is used to predict the masked or replaced part based on the context. The search text sequence is input into the masking layer, such as... Figure 2 As shown, the mask vector of the search word segmentation sequence is obtained.

[0070] S14. Input the token vector, mask vector, and segment vector into the encoder layer for feature extraction to obtain the text features corresponding to the search word segmentation sequence, the first word features corresponding to the search word segmentation, and the character features corresponding to the characters of the search word segmentation; wherein, the encoder layer is a multi-layer Transformer structure.

[0071] The three different embedding results—token vector, mask vector, and segment vector—are combined and used as the input to the encoder layer of the pre-trained model (which could be a BERT model). Bert ={E token E segment E InputMask The feature extraction process is performed through the encoder layer to obtain the feature extraction results. As an optional implementation, if the search text has only one search term, the segment vector can be omitted.

[0072] like Figure 2 As shown, the feature extraction results include the text features corresponding to the search word segmentation sequence (i.e., Figure 2 H CLS ), search for the first word feature corresponding to the word segmentation (i.e. Figure 2 of ) and the character features corresponding to the characters in the search segmentation (i.e. Figure 2 of H CLS It integrates the overall features (i.e., text features) of the search word segmentation sequence, that is and Features This integrates the overall features of each search word segment (i.e., the features of the first word), that is, the features of the i-th search word segment. The characteristics of H. CLS , and The features are combined to obtain the output, which contains the contextual semantics of the search word segmentation sequence. Bert : Where m is the number of search terms in the search term sequence, N represents the number of characters in the search term sequence, and i represents the i-th search term.

[0073] S15. Aggregate the character features based on the segment vector to obtain the second word features of the search segment.

[0074] In this embodiment, firstly, the position code E is used. segment ={E0,E0,…,E0,E1,…,E1,…,E i ,…,E m E m}, to obtain the character fragments corresponding to each search term, for example W i The length of the i-th search term is l. i The character features are then aggregated from the character fragments to obtain the second word features corresponding to the i-th search word.

[0075] As an optional implementation method, the aggregation processing can be done by max-pooling, min-pooling, mean-pooling, etc. Among them, max-pooling has the best effect. Of course, this embodiment is not limited to the above-mentioned aggregation processing methods.

[0076] like Figure 2 As shown, this aggregation process can be understood as a word-to-word conversion process, that is, aggregating the word features (i.e., second word features) of the corresponding search word by using the character features of the character segments of each search word. The second word features are... Figure 2 In

[0077] S16. Using text features, first word features, and second word features as inputs to the fully connected layer and real class tags as outputs, train the network parameters of the pre-trained model to obtain the search term category recognition model.

[0078] The fully connected layer is activated by the sigmoid function. As an optional implementation, text features and first-word features are used to characterize the contextual semantic information of each search word, serving as auxiliary features for category recognition. Second-word features, on the other hand, serve as the primary features for category recognition and play a dominant role in training. Alternatively, multiple search words are processed in parallel for classification, yielding category results for each search word. These category results include the category matched by the search word and the matching probability of that category.

[0079] like Figure 2 As shown, based on the second word feature For example, After entering the fully connected layer, we obtain the search word segmentation W. i Output value Calculate category losses based on actual category tags. After activation by the sigmoid function, the word segmentation W can be obtained. i Probability distribution across multiple categories Where K represents the number of categories. Specifically, the probability distribution... in, To search for the category corresponding to the word segmentation, and These are trainable weight and bias parameters. The loss function of the entire pre-trained model is backpropagated to continuously adjust the model. And related network parameters.

[0080] As an optional implementation, the Adam (Adaptive Moment Estimation) optimization algorithm is used to minimize the loss, and the pre-trained model is iteratively trained continuously. Finally, the network parameters obtained from the training are saved, and a search term category recognition model is trained.

[0081] This embodiment performs category identification for search terms based on the entire search text. It uses a search term category identification model to obtain the semantic information of the search text, the semantic information of each search term, and the semantic information of each character. For each search term, character features encompassing contextual features are extracted, and word features are obtained by aggregating character features. Then, based on the contextual and word features of each search term, the corresponding category and its probability are determined. This embodiment, by combining a search term category identification model with contextual and word features, determines the probability of category affiliation for words containing multiple categories, significantly improving the ambiguity problem in category identification and achieving higher accuracy.

[0082] Example 2

[0083] This embodiment provides a method for identifying search term categories, such as Figure 3 As shown, this search term category identification method includes the following steps:

[0084] S21. Obtain the search text to be recognized.

[0085] S23. Preprocess the search text to be identified to obtain the search word segmentation sequence to be identified, which includes multiple search words to be identified.

[0086] In this embodiment, the specific operations for preprocessing the search text to be identified are the same as those for preprocessing the search text in Embodiment 1.

[0087] S24. Input the search word segmentation sequence to be identified into the search word category identification model to obtain at least one category result corresponding to each search word to be identified.

[0088] The search term category recognition model is trained using the training method of the search term category recognition model in Example 1. As an optional implementation, the number of categories K to be retained can be specified. Taking "Lijiang Shangri-La" as an example, when K=1, the obtained category results may be ["City|0.98", "City|0.95"], and when K=2, the obtained category results may be {["City|0.98", "City|0.95"], ["Scenic Spot|0.88", "Hotel|0.75"]}.

[0089] In this embodiment, the search term category recognition method for search term category recognition is obtained through the training method of the search term category recognition model in Embodiment 1. When the search term category recognition model is applied to a specific category recognition task, its output category recognition results will be more accurate, and it can realize multi-category recognition of each search word in the entire search text in parallel, which is more efficient.

[0090] Example 3

[0091] This embodiment provides a training device for a search term category recognition model, such as... Figure 4 As shown, the training system for the search term category recognition model includes a first acquisition module 11, a model building module 12, an input conversion module 13, a feature extraction module 14, a word conversion module 15, and a training module 16.

[0092] The first acquisition module 11 is used to acquire multiple training samples; wherein, each training sample includes a search word segmentation sequence corresponding to the search text and a real class target tag corresponding to each search word in the search word segmentation sequence.

[0093] In this embodiment, the data source for the training samples is mainly search text, and the first acquisition module 11 needs to acquire multiple search texts from the target industry. For ease of explanation, the target industry is set as an industry related to tourism, although this embodiment is not limited to tourism scenarios. As an optional implementation, the first acquisition module 11 acquires the search text entered by the user in the front-end search box from a tourism-related search engine, and then preprocesses the search text, including data cleaning, Chinese word segmentation, and constructing a search word segmentation sequence.

[0094] See Figure 2The first acquisition module 11 performs data cleaning on the user-input search text to obtain unsegmented search text. Data cleaning includes operations such as removing irrelevant symbols, converting traditional Chinese characters to simplified Chinese characters, deleting extra spaces, and limiting length. For example, if the input search text is "Lijiang-Shangri-La", the unsegmented search text obtained after data cleaning will be "Lijiang Shangri-La".

[0095] As an optional implementation, the first acquisition module 11 matches the unsegmented search text with a thesaurus related to the target industry, and obtains the corresponding search terms from the thesaurus. For example, segmenting "Lijiang Shangri-La" yields search terms including "Lijiang", "Shangri-La", and "Shangri-La". Then, based on the above search terms, multiple search term sequences are generated. For example, the search term sequence obtained based on "Lijiang", "Shangri-La", and "Shangri-La" includes "Lijiang Shangri-La" and "Lijiang Shangri-La". Finally, the first acquisition module 11 selects the optimal search term sequence from the above search term sequences as a training sample.

[0096] As an optional implementation, when selecting the optimal search word segmentation sequence, the first acquisition module 11 compares the path span of the search word segmentation sequences, and usually selects the search word segmentation sequence with a larger path span as the optimal search word segmentation sequence, where path refers to the text length of each search word. As another optional implementation, when the path span is consistent, the first acquisition module 11 can obtain the word length of each search word, and select the search word segmentation sequence with the longer word length that is relatively later in its search word segmentation sequence as the optimal search word segmentation sequence, or select the search word segmentation sequence with more average word lengths as the optimal search word segmentation sequence. For example, assuming that "Lijiang Shangri-La" is segmented into "Lijiang Shangri-La" and "Lijiang Shangri-La", where "Shangri-La" is later in the sequence, the first acquisition module 11 selects "Lijiang Shangri-La" as the optimal search word segmentation sequence, and then uses "Lijiang Shangri-La" as a training sample.

[0097] The model building module 12 is used to build a pre-trained model, which includes an embedding layer, an encoder layer, and a fully connected layer.

[0098] As an optional implementation, the pre-training module of this embodiment is an improved BERT (Bidirectional Encoder Representation Based on Transformer, a pre-trained language representation model) model. BERT is pre-trained using MLM (Mask Language Model) and employs a deep bidirectional Transformer (encoder) component to generate deep bidirectional language representations, so that the output of the BERT model covers contextual semantic information.

[0099] The input conversion module 13 is used to input the search word segmentation sequence into the embedding layer to obtain the token vector, segment vector and mask vector; wherein, the embedding layer includes the token embedding layer, the segment embedding layer and the mask layer.

[0100] The embedding layer is used to convert the input search word segmentation sequence into a vector representation. The input conversion module 13 inputs the search word segmentation sequence into the token embedding layer, the segment embedding layer and the mask layer respectively to obtain the token vector, the segment vector and the mask vector.

[0101] In this embodiment, when the input conversion module 13 inputs the search word segmentation sequence into the token embedding layer, it also needs to input CLS and SEP tokens. As an optional implementation, [CLS] is inserted at the beginning of the search word segmentation sequence, and [SEP] is inserted at the end of each search word segment. After passing through the token embedding layer of the pre-trained model, the corresponding token vector is obtained, such as... Figure 2 As shown, the token vector corresponding to [CLS] is E. CLS The token vector corresponding to [SEP] is E. SEP The token vector corresponding to each character in the search word segmentation sequence is: This yields the token vector of the search word segmentation sequence. Where N is the number of characters in the search word segmentation sequence, i represents the i-th search word, and l i C represents the number of characters in the i-th search term. N This represents the Nth character.

[0102] In this embodiment, the segment vector represents the position of the search word in the search text. The input conversion module 13 inputs the search word sequence into the segment embedding layer, such as... Figure 2 As shown, the segment vector E of the search word segment sequence is obtained. segment ={E0,E0,…,E1,…,E1,…,E i ,…,E m E m}, where m represents the number of search terms.

[0103] In this embodiment, the mask vector represents the random masking or replacement of any word or phrase in the search text. During training, the smallest input unit token in the entire sentence is randomly selected for masking or replacement, and then the pre-trained model is used to predict the masked or replaced part based on the context. The input conversion module 13 inputs the search text sequence into the masking layer, such as... Figure 2 As shown, the mask vector of the search word segmentation sequence is obtained.

[0104] The feature extraction module 14 is used to input the token vector, mask vector and segment vector into the encoder layer for feature extraction, to obtain the text features corresponding to the search word segmentation sequence, the first word features corresponding to the search word segmentation and the character features corresponding to the characters of the search word segmentation; wherein, the encoder layer is a multi-layer Transformer structure.

[0105] Feature extraction module 14 combines the three different embedding results—token vector, mask vector, and segment vector—as the input to the encoder layer of the pre-trained model (which could be a BERT model). Bert ={E token E segment E InputMask The feature extraction process is performed through the encoder layer to obtain the feature extraction results. As an optional implementation, if the search text has only one search term, the segment vector can be omitted.

[0106] like Figure 2 As shown, the feature extraction results include the text features corresponding to the search word segmentation sequence (i.e., Figure 2 H CLS ), search for the first word feature corresponding to the word segmentation (i.e. Figure 2 of ) and the character features corresponding to the characters in the search segmentation (i.e. Figure 2 of H CLS It integrates the overall features (i.e., text features) of the search word segmentation sequence, that is and Features This integrates the overall features of each search word segment (i.e., the features of the first word), that is, the features of the i-th search word segment. Its characteristics.

[0107] Feature extraction module 14 will extract H CLS , and The features are combined to obtain the output, which contains the contextual semantics of the search word segmentation sequence. Bert : Where m is the number of search terms in the search term sequence, N represents the number of characters in the search term sequence, and i represents the i-th search term.

[0108] The word conversion module 15 is used to aggregate word features based on segment vectors to obtain the second word features of the search segment. In this embodiment, the word conversion module 15 first uses positional encoding Esegment ={E0,E0,…,E0,E1,…,E1,…,E i ,…,E m E m}, to obtain the character fragments corresponding to each search term, for example W i The length of the i-th search term is l. i The character features are then aggregated by the word conversion module 15 to obtain the second word features corresponding to the i-th search word.

[0109] As an optional implementation method, the aggregation processing can be done by max-pooling, min-pooling, mean-pooling, etc. Among them, max-pooling has the best effect. Of course, this embodiment is not limited to the above-mentioned aggregation processing methods.

[0110] like Figure 2 As shown, this aggregation process can be understood as a word-to-word conversion process, that is, aggregating the word features (i.e., second word features) of the corresponding search word by using the character features of the character segments of each search word. The second word features are... Figure 2 In

[0111] Training module 16 is used to train the network parameters of the pre-trained model with text features, first word features and second word features as inputs to the fully connected layer and real class tags as outputs, so as to obtain the search term category recognition model.

[0112] The fully connected layer is activated by the sigmoid function. As an optional implementation, text features and first-word features are used to characterize the contextual semantic information of each search word, serving as auxiliary features for category recognition. Second-word features, on the other hand, serve as the primary features for category recognition and play a dominant role in training. Alternatively, multiple search words are processed in parallel for classification, yielding category results for each search word. These category results include the category matched by the search word and the matching probability of that category.

[0113] like Figure 2 As shown, based on the second word feature For example, After entering the fully connected layer, we obtain the search word segmentation W. i Output value Training module 16 calculates the category loss based on the real category tags. After activation by the sigmoid function, the word segmentation W can be obtained. i Probability distribution across multiple categories Where K represents the number of categories. Specifically, the probability distribution... in, To search for the category corresponding to the word segmentation, and These are trainable weight and bias parameters. The loss function of the entire pre-trained model is backpropagated to continuously adjust the model. And related network parameters.

[0114] As an optional implementation, the training module 16 uses the Adam (Adaptive Moment Estimation) optimization algorithm to minimize the loss and continuously iterates the training of the pre-trained model. Finally, the training module 16 saves the network parameters obtained from the training and trains the search term category recognition model.

[0115] Example 4

[0116] This embodiment provides a search term category recognition device, such as... Figure 5 As shown, the search term category recognition device includes a second acquisition module 21, a preprocessing module 22, and a category recognition module 23.

[0117] The second acquisition module 21 is used to acquire the search text to be recognized.

[0118] The preprocessing module 22 is used to preprocess the search text to be identified, obtaining a search word segmentation sequence to be identified, which includes multiple search words to be identified. In this embodiment, the specific operation of the preprocessing module 22 in preprocessing the search text to be identified is the same as the specific operation in embodiment 3 for preprocessing the search text.

[0119] The category recognition module 23 is used to input the search word segmentation sequence to be recognized into the search word category recognition model, and obtain at least one category result corresponding to each search word to be recognized. The search word category recognition model is trained using the training method of the search word category recognition model in Example 1. As an optional implementation, the category recognition module 23 can specify the number of categories to retain, K. Taking "Lijiang Shangri-La" as an example, for instance, when K=1, the obtained category result might be ["City|0.98", "City|0.95"], or when K=2, the obtained category result might be {["City|0.98", "City|0.95"], ["Scenic Spot|0.88", "Hotel|0.75"]}.

[0120] Example 3

[0121] This embodiment provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the training method of the search term category recognition model of Embodiment 1 or the search term category recognition method of Embodiment 2.

[0122] like Figure 6 The electronic device 30 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0123] The electronic device 30 may be in the form of a general-purpose computing device, such as a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including memory 32 and processor 31).

[0124] Bus 33 includes a data bus, an address bus, and a control bus.

[0125] The memory 32 may include volatile memory, such as random access memory (RAM) 321 and cache memory 322, and may further include read-only memory (ROM) 323.

[0126] The memory 32 may also include a program tool 325 having a set (at least one) of program modules 324, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0127] The processor 31 executes various functional applications and data processing by running computer programs stored in the memory 32, such as the training method of the search term category recognition model in Embodiment 1 of the present invention or the search term category recognition method in Embodiment 2.

[0128] Electronic device 30 can also communicate with one or more external devices 34. This communication can be performed via input / output (I / O) interface 35. Furthermore, model generation device 30 can also communicate with one or more networks via network adapter 36. Figure 6 As shown, network adapter 36 communicates with other modules of the model-generating device 30 via bus 33. It should be understood that, although... Figure 6 Unless otherwise specified, the device 30 generated in conjunction with the model may use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

[0129] It should be noted that although several units / modules or sub-units / modules of the electronic device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0130] Example 4

[0131] This embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the training method of the search term category recognition model of Embodiment 1 or the search term category recognition method of Embodiment 2.

[0132] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0133] In an alternative embodiment, the present invention can also be implemented as a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the terminal device to execute the training method for the search term category recognition model of Embodiment 1 or the search term category recognition method of Embodiment 2.

[0134] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of the present invention is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but all such changes and modifications fall within the scope of protection of the present invention.

Claims

1. A training method for a search term category recognition model, characterized in that, The training method for the search term category recognition model includes: Obtain multiple training samples; wherein each training sample includes a search word segmentation sequence corresponding to the search text and a real class target tag corresponding to each search word in the search word segmentation sequence; Construct a pre-trained model, which includes an embedding layer, an encoder layer, and a fully connected layer; The search word segmentation sequence is input into the embedding layer to obtain a token vector, a segment vector, and a mask vector; wherein, the embedding layer includes a token embedding layer, a segment embedding layer, and a mask layer; The token vector, the mask vector, and the segment vector are input into the encoder layer for feature extraction to obtain the text features corresponding to the search word segmentation sequence, the first word features corresponding to the search word segmentation, and the character features corresponding to the characters of the search word segmentation; wherein, the encoder layer is a multi-layer Transformer structure; Based on the segment vector, the character features are aggregated to obtain the second word features of the search word segmentation; Using the text features, the first word features, and the second word features as inputs to the fully connected layer, and the real class target tags as outputs, the network parameters of the pre-trained model are trained to obtain the search term category recognition model.

2. The training method for the search term category recognition model according to claim 1, characterized in that, The step of inputting the search word segmentation sequence into the embedding layer to obtain the token vector, segment vector, and mask vector includes: The token embedding layer inserts a CLS token at the beginning of the search word segmentation sequence and a SEP token at the end of each search word, mapping the CLS token, the SEP token, and the characters of the search word segmentation sequence to a token vector. The segment embedding layer encodes and maps the position of the search word in the search word sequence into a segment vector. The characters in the search word segmentation sequence are replaced sequentially using the MASK marker through the mask layer, and the replaced search word segmentation sequence is mapped to a mask vector.

3. The training method for the search term category recognition model according to claim 1, characterized in that, The step of aggregating the character features based on the segment vector includes: Based on the segment vector, obtain the character fragments corresponding to each search word; The character features under the character fragment are aggregated to obtain the second word features corresponding to the search word segment.

4. The training method for the search term category recognition model according to claim 1, characterized in that, The step of obtaining multiple training samples is preceded by: Retrieve multiple search texts from the target industry; For each search text, the search text is preprocessed to obtain the training samples.

5. The training method for the search term category recognition model according to claim 4, characterized in that, The preprocessing steps for the search text include: The search text is cleaned; the data cleaning includes filtering out meaningless symbols and removing spaces. The search text after data cleaning is matched with a thesaurus related to the target industry to obtain multiple search terms; Multiple search word segments are generated based on the search word segmentation connections; The optimal search word segmentation sequence is selected from the search word segmentation sequence as the training sample.

6. A method for identifying search term categories, characterized in that, The steps of the search term category identification method include: Obtain the search text to be recognized; The search text to be identified is preprocessed to obtain a search word segmentation sequence to be identified, which includes multiple search words to be identified. The search word segmentation sequence to be identified is input into the search word category recognition model to obtain at least one category result corresponding to each search word to be identified. The search term category recognition model is obtained by the training method of the search term category recognition model as described in any one of claims 1-5.

7. A training device for a search term category recognition model, characterized in that, The fully connected layer of the search term category recognition model includes multiple parallel fully connected layers, and the training device for the search term category recognition model includes: The first acquisition module is used to acquire multiple training samples; wherein, the training samples include a search word segmentation sequence corresponding to the search text and a real class target tag corresponding to each search word in the search word segmentation sequence; The model building module is used to build a pre-trained model, which includes an embedding layer, an encoder layer, and a fully connected layer. The input conversion module is used to input the search word segmentation sequence into the embedding layer to obtain a token vector, a segment vector, and a mask vector; wherein, the embedding layer includes a token embedding layer, a segment embedding layer, and a mask layer; The feature extraction module is used to input the token vector, the mask vector, and the segment vector into the encoder layer for feature extraction, to obtain the text features corresponding to the search word segmentation sequence, the first word features corresponding to the search word segmentation, and the character features corresponding to the characters of the search word segmentation; wherein, the encoder layer is a multi-layer Transformer structure; The word conversion module is used to aggregate the word features based on the segment vector to obtain the second word features of the search word segmentation; The training module is used to train the network parameters of the pre-trained model with the text features, the first word features, and the second word features as inputs to the fully connected layer and the real class tags as outputs, so as to obtain the search term category recognition model.

8. A search term category recognition device, characterized in that, The search term category recognition device includes: The second acquisition module is used to acquire the search text to be recognized; The preprocessing module is used to preprocess the search text to be identified to obtain a search word segmentation sequence to be identified, wherein the search word segmentation sequence to be identified includes multiple search words to be identified; The category recognition module is used to input the search word segmentation sequence to be recognized into the search word category recognition model to obtain at least one category result corresponding to each search word to be recognized. The search term category recognition model is obtained through the training device for the search term category recognition model as described in claim 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the training method of the search term category recognition model as described in any one of claims 1-5 or the search term category recognition method as described in claim 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the training method of the search term category recognition model as described in any one of claims 1-5 or the search term category recognition method as described in claim 6.

Images