Information classification method and device and server

Through the multi-grained matching attention network model, combined with self-match and semantic level matching modules, the multi-grained features of query information are extracted, which solves the problem of insufficient generalization ability of query intention classification models in the existing technology, and improves the e-commerce search system Query intent classification accuracy and recall ability.

CN118939797APending Publication Date: 2024-11-12BEIJING WODONG TIANJUN INFORMATION TECH CO LTD +1
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202310521636.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, the generalization ability of the query intent classification model is insufficient, resulting in a decrease in the classification accuracy of query information. Especially in the e-commerce search system, short query statements and huge object categories lead to increased classification difficulty and lack of long-tail queries. When the information is sufficiently supervised, the model cannot effectively capture the user's true intentions.

Method used

Using a multi-grained matching attention network model, combining self-match module, character-level matching module and semantic matching module, fine-grained and coarse-grained features of query information are extracted, and the query and category are captured through convolution processing and cross-attention processing. multi-grained matching features between them enhance representation learning and semantic disambiguation capabilities.

Benefits of technology

It improves the classification accuracy of query information, enhances the model's ability to recall long-tail queries and multi-intents, makes up for the expression gap between query and categories, and improves the classification effect of query intentions in the e-commerce search system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939797A_ABST
    Figure CN118939797A_ABST
Patent Text Reader

Abstract

The invention relates to an information classification method and device and a server, and relates to the technical field of computers. The information classification method comprises the following steps: extracting query information of a user and a first matching feature of each object category in a plurality of object categories by utilizing convolution processing; extracting the query information and a second matching feature of each object category by using cross attention processing; and determining an object category corresponding to the query information according to the first matching feature and the second matching feature. According to the technical scheme, the generalization ability of classification processing can be improved, so that the classification accuracy of information is inquired.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an information classification method, an information classification device, a server, and a non-volatile computer-readable storage medium. Background Art

[0002] E-commerce platforms provide consumers with hundreds of millions of products. A typical e-commerce search system usually consists of the following modules: query parsing, item recall, relevance filtering, and item ranking.

[0003] Query parsing consists of word segmentation, error correction, query rewriting, named entity recognition, and user intent detection. As an important part of it, query intent classification helps the recall and relevance modules filter out irrelevant items and improve the user's shopping experience. In addition, the predicted category of the user's query can be used as a feature for item sorting to help users quickly find the most relevant items. Therefore, the query intent classification task has attracted more and more attention from academia and industry, and improving the accuracy of intent classification is crucial for retrieval systems.

[0004] In related technologies, contextual information of documents is learned through a multi-label classification model to enhance query representation learning. Summary of the invention

[0005] The inventors of the present disclosure have discovered that the above-mentioned related technologies have the following problems: the generalization capability of the classification process is insufficient, resulting in a decrease in the classification accuracy of the query information.

[0006] In view of this, the present disclosure proposes an information classification technology solution, which can improve the generalization ability of classification processing and thus improve the classification accuracy of query information.

[0007] According to some embodiments of the present disclosure, there is provided an information classification method, comprising: utilizing convolution processing to extract first matching features between a user's query information and each of a plurality of object categories; utilizing cross-attention processing to extract second matching features between the query information and each of the object categories; and determining the object category corresponding to the query information based on the first matching features and the second matching features.

[0008] In some embodiments, extracting the first matching feature for the query information and each of the plurality of object categories includes: extracting the first matching feature according to a dot product processing result of the query feature of the query information and the category feature of each object category.

[0009] In some embodiments, extracting the first matching feature includes: calculating a dot product processing result based on query information, a transposition of the category feature, and a weight parameter, wherein the weight parameter is determined by training a first machine learning model.

[0010] In some embodiments, extracting the first matching feature includes: performing two-dimensional convolution processing on the dot product processing results on multiple channels to determine a convolution feature map; and extracting the first matching feature based on the convolution feature map.

[0011] In some embodiments, extracting the first matching feature includes: performing maximum pooling processing on the convolution feature map to extract the first matching feature.

[0012] In some embodiments, extracting the query information and the second matching features of each object category includes: performing mean pooling processing on the category features of each object category at the time step of the relevant information of each object category to determine the mean pooling processing result; and extracting the second matching features based on the mean pooling processing result and the query features of the query information.

[0013] In some embodiments, extracting the first matching feature between the user's query information and each object category in a plurality of object categories includes: extracting the category feature of each object category based on the relevant information of each object category and the relevant information of the objects belonging to the object category; extracting the query feature of the query information based on at least one word included in the query information; extracting the first matching feature based on the query feature and the category feature; extracting the second matching feature between the query information and each object category includes: extracting the second matching feature based on the query feature and the category feature.

[0014] In some embodiments, the relevant information of the object category includes name information of the object category, and the relevant information of the object includes keywords in the description information of the object.

[0015] In some embodiments, the information classification method also includes: extracting query features of the query information based on at least one word contained in the query information; extracting self-attention features of the query information using self-attention processing based on the query features; wherein determining the object category corresponding to the query information includes: determining the object category corresponding to the query information based on the self-attention features.

[0016] In some embodiments, determining the object category corresponding to the query information includes: determining the object category corresponding to the query information using nonlinear transformation processing based on the first matching feature, the second matching feature, and the self-attention feature.

[0017] In some embodiments, the object category includes at least one of an item category or a service category, and determining the object category corresponding to the query information includes: when the query information involves an object of a service category, determining the service category corresponding to the query information based on a first matching feature and a second matching feature and using a first classification module of a first machine learning model.

[0018] In some embodiments, the information classification method also includes: determining the item category corresponding to the query information using a second machine learning model based on query features of the query information and category features of each object category, the second machine learning model being different from the first machine learning model; and when the query information involves objects of a service category, determining the classification result of the query information based on the service category corresponding to the query information and the item category corresponding to the query information.

[0019] In some embodiments, the information classification method further includes: using a second classification module of the first machine learning model to determine whether the query information involves an object of a service category.

[0020] In some embodiments, the first matching feature is extracted by a convolution module of a first machine learning model, the second matching feature is extracted by a cross-attention module of the first machine learning model, the object category corresponding to the query information is determined by a first classification module of the first machine learning model, and the first machine learning model is trained by a multi-label cross entropy loss function, which is calculated based on the object category corresponding to the query information sample and the annotation results of the query information sample.

[0021] According to some other embodiments of the present disclosure, there is provided an information classification device, comprising: a convolution module for extracting a first matching feature between a user's query information and each of a plurality of object categories; a cross-attention module for extracting a second matching feature between the query information and each of the object categories; and a first classification module for determining an object category corresponding to the query information based on the first matching feature and the second matching feature.

[0022] In some embodiments, the convolution module extracts the first matching feature according to a dot product processing result of a query feature of the query information and a category feature of each object category.

[0023] In some embodiments, the convolution module calculates the dot product processing result based on the query information, the transposition of the category features and the weight parameters, and the weight parameters are determined by training the first machine learning model.

[0024] In some embodiments, the convolution module performs two-dimensional convolution processing on the dot product processing results on multiple channels to determine a convolution feature map; and extracts the first matching feature based on the convolution feature map.

[0025] In some embodiments, the convolution module performs maximum pooling processing on the convolution feature map to extract the first matching feature.

[0026] In some embodiments, the cross-attention module performs mean pooling on the category features of each object category at the time step of the relevant information of each object category to determine the mean pooling result; and extracts the second matching feature based on the mean pooling result and the query feature of the query information.

[0027] In some embodiments, the information classification device also includes: a feature extraction module, which is used to extract the category features of each object category based on the relevant information of each object category and the relevant information of the objects belonging to the object category, and extract the query features of the query information based on at least one word contained in the query information; a convolution module, which extracts the first matching feature based on the query feature and the category feature; and a cross-attention module, which extracts the second matching feature based on the query feature and the category feature.

[0028] In some embodiments, the relevant information of the object category includes name information of the object category, and the relevant information of the object includes keywords in the description information of the object.

[0029] In some embodiments, the information classification device also includes: a feature extraction module, used to extract query features of the query information based on at least one word contained in the query information; a self-attention module, used to extract self-attention features of the query information based on the query features; and a first classification module, which determines the object category corresponding to the query information based on the self-attention features.

[0030] In some embodiments, the first classification module determines the object category corresponding to the query information using nonlinear transformation processing based on the first matching feature, the second matching feature and the self-attention feature.

[0031] In some embodiments, the object category includes at least one of an item category or a service category. When the query information involves an object of the service category, the first classification module determines the service category corresponding to the query information based on the first matching feature and the second matching feature.

[0032] In some embodiments, the information classification device includes a first machine learning model, which includes a convolution module, a cross-attention module, and a first classification module.

[0033] In some embodiments, the information classification device also includes: a second machine learning model, used to determine the item category corresponding to the query information based on the query characteristics of the query information and the category characteristics of each object category, the second machine learning model is different from the first machine learning model; in the case where the query information involves objects of the service category, the classification result of the query information is determined based on the service category corresponding to the query information and the item category corresponding to the query information.

[0034] In some embodiments, the information classification device further includes: a second classification module, configured to determine whether the query information involves an object of a service category.

[0035] In some embodiments, the first machine learning model is trained by a multi-label cross entropy loss function, which is calculated based on the object category corresponding to the query information sample and the annotation results of the query information sample.

[0036] According to some further embodiments of the present disclosure, there is provided an information classification device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the information classification method in any of the above embodiments based on instructions stored in the memory device.

[0037] According to some further embodiments of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the information classification method in any of the above embodiments is implemented.

[0038] According to some further embodiments of the present disclosure, a server is provided, comprising: the information classification device in any one of the above embodiments.

[0039] In the above embodiment, convolution processing is used to capture the fine-grained matching features between the query information and the object category; cross-attention processing is used to capture the coarse-grained matching features of the fine-grained matching features between the query information and the object category; and the query information is classified according to the fine-grained matching features and the coarse-grained matching features. In this way, the generalization ability of the classification processing can be improved, thereby improving the classification accuracy of the query information. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0041] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings:

[0042] Figure 1 A flowchart showing some embodiments of the information classification method of the present disclosure;

[0043] Figure 2 Schematic diagrams showing some embodiments of the multi-granularity matching attention network structure of the present disclosure;

[0044] Figure 3 A schematic diagram showing some embodiments of the online service architecture of the present disclosure;

[0045] Figure 4 A block diagram showing some embodiments of the information classification device of the present disclosure;

[0046] Figure 5 A block diagram showing some other embodiments of the information classification device disclosed herein;

[0047] Figure 6 The following is a block diagram showing some further embodiments of the information classification device disclosed herein. DETAILED DESCRIPTION

[0048] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0049] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0050] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0051] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0052] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0053] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0054] As mentioned above, with the increase in user demand, non-physical products such as "services (such as consulting, etc.)" have also appeared on the platform. Due to the diversity of user needs and item types, e-commerce search systems need to have the ability to identify search query intent. With the development of deep learning, query intent classification applies more natural language processing techniques to replace rule-based methods.

[0055] In some scenarios, query information is usually short, lacks sufficient contextual information, and is even insensitive to word order, such as e-commerce application scenarios. These technical issues prevent models such as XML-CNN (eXtreme Multi-Label-Convolutional Neural Networks), HiAGM (Hierarchy-Aware Global Model), and LSAN (Label Specific Attention Network) from capturing the most important information in user-intent classification queries. In addition, queries may have different requirements, and these polysemous queries increase the difficulty of accurately classifying user intent.

[0056] That is to say, on the one hand, the above models have insufficient generalization capabilities for users’ long-tail queries. For example, unlike general search engines, queries for e-commerce searches are usually very short and colloquial, while the corresponding object categories are very large, which is a challenging task for industry applications. For long-tail queries that lack sufficient supervisory information from users, this technical problem becomes even more serious.

[0057] The above classification models for query intent either design more sophisticated models to enhance query representation learning or explore label graphs and multi-tasks to facilitate the model to learn external information. The structures of these models are relatively simple, and the number of parameters is not enough to learn enough knowledge. These models cannot capture multi-granular matching features from queries and categories, which makes it difficult for them to bridge the expression gap between informal queries and categories.

[0058] On the other hand, the above model is not capable of recalling multiple intents of user queries. For example, due to different user backgrounds, the query sentences entered by users may contain multiple intents or generalized requirements. The above model cannot capture the multiple intents of such ambiguous expressions, resulting in incomplete intent recall.

[0059] In response to the above technical problems, the present disclosure proposes a multi-granularity matching attention network model, which includes modules such as a self-matching module, a character-level matching module, and a semantic-level matching module, so as to comprehensively extract the features of the query itself and the query-category interaction features.

[0060] In this way, the network model can eliminate the expression differences between queries and categories to prompt the effect of query intent classification.

[0061] For example, the present disclosure proposes a new strategy for explicitly extending category information to reduce the expression gap between queries and categories, which includes a self-matching module, a character-level matching model, and a semantic-level matching model, respectively focusing on query representation learning, long-tail query enhancement, and semantic disambiguation.

[0062] For example, the query and category representation learning module of the present invention describes the mapping process of query or category sequence from word vector to the same semantic space; the self-matching module explains how to learn a good query semantic representation only by querying its own character sequence; the character-level matching module performs query and category matching in the character vector space to capture fine-grained matching features between the two; and the semantic-level matching module measures the semantic similarity between query and category representation to capture coarse-grained matching features.

[0063] For example, the three feature representations of queries and categories learned by the three matching modules can be concatenated and provided as input features to the classifier to predict the query intent.

[0064] Figure 1 A flowchart showing some embodiments of the information classification method of the present disclosure.

[0065] like Figure 1 As shown, in step 110, a convolution process is used to extract the first matching feature between the user's query information and each of the multiple object categories.

[0066] In some embodiments, based on the relevant information of each object category and the relevant information of objects belonging to the object category, the category features of each object category are extracted; based on at least one word included in the query information, the query features of the query information are extracted; based on the query features and the category features, the first matching features and the second matching features are extracted.

[0067] For example, we first need to extract the vector representation of the query information and each object category as the query feature and category feature respectively. Vector representation of the query information and each object category is the basis for aligning the two to the same semantic space. Pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers) can be selected as encoders for query information and each object category.

[0068] In some embodiments, the relevant information of the object category includes name information or attribute information of the object category, and the relevant information of the object includes description information of the object or keywords in the attribute information. For example, the relevant information of the object may also include average Embedding feature information of the object.

[0069] For example, for long-tail user queries, there is often a lack of sufficient training samples to accurately predict the user's intention. In this case, auxiliary knowledge such as the name information of the category and the core item words of the object (i.e., the keywords in the description information) can be used to assist the model in making classification decisions.

[0070] For example, in order to better learn the semantics of an object category, the character sequence of the object category consists of two parts: the name information of the object category n = [n1, n2, ..., n j ,…,n Ln ],n j represents the jth object category, L n Represents the total number of object categories; the core product words of each object contained in the jth object category L m Indicates the number of core product words.

[0071] In some embodiments, the description information of the object may include the title information of the object, and the core product word may be at least one keyword pre-marked in the title information.

[0072] For example, the title information of an object is relatively long and may contain multiple product words. The core product word is the most important among these words. For example, the title information of an object is "xxxxx milk grape xxxx", which contains two product words "milk" and "grape", and the core product word is "grape".

[0073] In some embodiments, after obtaining the core product word Then, they are combined with the name information of the object category to which they belong [n j ] are connected to generate the category representation Will Input the feature extraction module BERT of the BERT model token (), encode the representation of the object category to extract the category features of the object category.

[0074] For example, in order to map query information and object categories into the same semantic space, query information and object categories can share the same BERT model.

[0075] For example, the query feature Q of query information i i and the category feature C of object category j j It can be expressed as follows:

[0076]

[0077]

[0078] BERT token () can be the last layer of the BERT model with the classification layer CLS removed; The token embedding matrices that can represent query information and object categories respectively.

[0079] In some embodiments, the first matching feature is extracted based on a dot product processing result of a query feature of the query information and a category feature of each object category. For example, the dot product processing result is calculated based on the query information, the transposition of the category feature, and a weight parameter, and the weight parameter is determined by training the first machine learning model.

[0080] For example, in order to extract fine-grained interaction features between query information and all object categories, we can i and category representation C j Then, we stack each query-category interaction matrix M on the channel dimension. j , which can be expressed as:

[0081]

[0082] M=[M1,M2,…,M j ,…,M Class ]

[0083] is a trainable weight parameter, is the interaction matrix between queries and categories, It is a feature map of character-level interactions between query information and each object category, and Class is the total number of object categories.

[0084] In some embodiments, two-dimensional convolution processing is performed on the dot product processing results on multiple channels to determine a convolution feature map; and the first matching feature is extracted based on the convolution feature map.

[0085] For example, in order to extract fine-grained matching features of each channel from the feature map M, a window size of r can be used. w × h The two-dimensional convolution module performs convolution operation on each channel of the feature map M as follows:

[0086]

[0087] σ(·) can be a ReLU activation function, k represents the kth channel of the feature map M, and W a,b is the convolution kernel, b is the bias vector, M i+a,j+b It is the i+a,j+bth element of the kth channel of the feature map.

[0088] In some embodiments, the convolutional feature map is subjected to maximum pooling processing to extract the first matching feature.

[0089] For example, a 2D max pooling layer can be applied to the convolutional feature map Extract most of the important features i,j as follows:

[0090]

[0091] p w and p h are the width and height of the 2D max pooling.

[0092] For example, the convolution feature map after the maximum pooling can be flattened (i.e., the matrix can be flattened to one dimension) and mapped to a low-dimensional space through a linear transformation layer, represented as the first matching feature The first matching feature contains the fine-grained matching features between the query information and each object category.

[0093] In the above embodiment, for long-tail queries, when the model lacks sufficient training samples to accurately predict the user's intention, auxiliary knowledge such as the name of the object category and the core product word is needed to assist the model in determining the classification decision. Therefore, the classification accuracy can be improved by extracting the first matching feature of the query information-object category as a fine-grained interaction feature.

[0094] For example, in e-commerce applications, the query information entered by users may contain typos or colloquial expressions, which are far from the official product titles and names of object categories. Therefore, the above embodiment learns the fine-grained similarity between query information and object categories at the character level, which is very important for the model to generalize grammatical queries and can improve classification accuracy.

[0095] In step 120, the query information and the second matching features of each object category are extracted using cross-attention processing.

[0096] In some embodiments, at the time step of the relevant information of each object category, the category features of each object category are average pooled to determine the average pooling result; and the second matching feature is extracted based on the average pooling result and the query feature of the query information.

[0097] For example, in order to obtain a category representation at the semantic level, the representation of the object category C j The time steps of the character sequence are mean pooled, and the representations of each mean-pooled object category are stacked together to determine the comprehensive representation of all object categories.

[0098] c j =mean(C j )

[0099]

[0100] For example, a criss-cross attention layer is applied to combine the query information with the comprehensive representation of all object categories to determine the second matching feature Z2:

[0101]

[0102] are trainable weights, It is a coarse-grained matching feature between query and category at the semantic level.

[0103] In the above embodiment, semantic-level interaction features of query information-object category are extracted. Because the words in the query information may be polysemous, relying solely on character-level matching may not be sufficient to fully capture the user's true intention. For example, "apple watch" and "apple juice" are highly correlated with "apple" at the character level, but they are different from each other at the semantic level. Therefore, in addition to considering the fine-grained matching features between the query information and the object type, the context-aware similarity between the query information and the object type, i.e., the coarse-grained matching features, is also captured at the semantic level to improve the classification accuracy.

[0104] In step 130, the object category corresponding to the query information is determined according to the first matching feature and the second matching feature.

[0105] In some embodiments, based on at least one word contained in the query information, query features of the query information are extracted; based on the query features, self-attention features of the query information are extracted using self-attention processing; and based on the self-attention features, an object category corresponding to the query information is determined.

[0106] For example, the features of pure query text can be extracted to classify query information. The classification processing of the text classification model can be built on pure query text, so the advantages of the self-attention mechanism can be used to extract text features of pure user query information, thereby improving the accuracy of classification.

[0107] For example, the query terms contained in the query information contribute differently to the entire query sentence vector, and an attention mechanism can be applied to summarize the query embedding matrix to extract intent-related query terms that are important for representing the query.

[0108] For example, we can use the query feature Q i , extract the self-attention feature q of the query information i :

[0109]

[0110]

[0111] They are the parameter vector and parameter matrix after random initialization, both of which are parameters that can be learned through machine learning model training.i,t refers to the matrix u i In the query information, the word x t The corresponding vector; Q i,t refers to the matrix Q i In the query information, the word x t The corresponding vector.

[0112] For example, α = softmax(u i,t ) is an evaluation function used to evaluate the importance of the query word for the sentence representation of the current query information. i Feeding into the self-attention module, we can obtain self-attention features using only our own information is used to predict the user's intention.

[0113] In some embodiments, the object category corresponding to the query information is determined using a nonlinear transformation process based on the first matching feature, the second matching feature, and the self-attention feature. For example, the first matching feature, the second matching feature, and the self-attention feature may be concatenated together to determine the object category corresponding to the query information.

[0114] For example, the query representation q i , both fine-grained query information-object category matching features Z1 and coarse-grained matching features Z2 help predict the user's intention, so they can be fused together using matrix multiplication.

[0115] For example, a nonlinear transformation layer can be introduced to i , Z1 and Z2 are fused to obtain the fused feature X:

[0116] X = ReLU(q i W qf +[Z1;Z2]W z )

[0117] are learnable weights.

[0118] In some embodiments, the prediction parameters of the query information for each object category can be determined based on the fusion features. As a classification result. For example, is a learnable parameter, Represents the prediction parameters of the query information for the object category c.

[0119] In some embodiments, the first matching feature is extracted by a convolution module of a first machine learning model, the second matching feature is extracted by a cross-attention module of the first machine learning model, the object category corresponding to the query information is determined by a first classification module of the first machine learning model, and the first machine learning model is trained by a multi-label cross entropy loss function, which is calculated based on the object category corresponding to the query information sample and the annotation results of the query information sample.

[0120] For example, the first machine learning model may also be trained using a circle loss function, an MCE (minimum classification error) loss function, or the like.

[0121] For example, you can use To represent the true classification label of the query information, y c ∈{0,1} indicates whether the query information belongs to the object category c. A multi-label cross entropy loss function can be used Train the machine learning model:

[0122]

[0123] σ(·) can be a sigmoid activation function.

[0124] In some embodiments, a second classification module of the first machine learning model is used to determine whether the query information involves an object of a service category.

[0125] For example, three models can be used to implement the online prediction process: a first machine learning model including a second classification module for intent prediction of long-tail queries; a first machine learning model including a first classification module for object category prediction of long-tail queries; and a second machine learning model for general query prediction (i.e., predicting item categories), such as the BERT model.

[0126] In some embodiments, when the query information involves an object of a service category, the service category corresponding to the query information is determined based on the first matching feature and the second matching feature using the first classification module of the first machine learning model.

[0127] In some embodiments, a second machine learning model is used to determine the item category corresponding to the query information based on query features of the query information and category features of each object category, and the second machine learning model is different from the first machine learning model; when the query information involves objects of service categories, the classification result of the query information is determined based on the service category corresponding to the query information and the item category corresponding to the query information.

[0128] In some embodiments, the feature extraction module of the first machine learning model can be a feature extraction module of a BERT model, a convolutional network, or a recurrent neural network, such as the last layer of a BERT model with the classification layer CLS removed; the second machine learning model can be a BERT model, a convolutional network, or a recurrent neural network.

[0129] For example, in order to reduce the response latency of online deployment, a 4-layer BERT model can be extracted from the 12-layer BERT-base as a text encoder (feature extraction module), and the rest of the modules of the 4-layer BERT model remain unchanged.

[0130] In some embodiments, for the application scenario of long-tail queries, a first machine learning model is first used to determine whether a query information belongs to a specified scenario (such as whether it involves an object of a service category); then, if it does, the first machine learning model is used to predict the service object category corresponding to the query information in the scenario; if it does not, the second machine learning model is used to predict the item object category corresponding to the query information.

[0131] For example, the scene categories (such as service object categories) can be merged with the prediction results of ordinary BERT serving general queries (such as item object categories) into the final classification results to improve the classification accuracy.

[0132] In some embodiments, at the beginning of the workflow, the system maintains a local cache module to directly store the object category prediction results of high-frequency user queries in memory; if the query does not hit the cache, the online prediction process will be triggered.

[0133] In the above embodiment, a multi-granularity matching attention network structure model is proposed. The various modules included in this model extract multi-granularity interaction features from the interaction matrix between user query information and object categories. Through the design of this model, the model can make up for the difference between user expression and object category. Through the prior information of the category, the technical problems of inaccurate prediction and incomplete recall caused by insufficient query information are made up.

[0134] Figure 2 Schematic diagrams showing some embodiments of the multi-granularity matching attention network structure of the present disclosure.

[0135] like Figure 2 As shown, the first machine learning model can include a multi-granularity matching attention network structure, which consists of four main modules: a query and category representation learning module (such as a feature extraction module); a self-matching module (self-attention module); a character-level matching module (such as a convolution module); and a semantic-level matching module (such as a cross-attention module).

[0136] In some embodiments, first, the query and category representation learning module describes the mapping process of the text sequence of query information or object category from word vector to semantic space; then, through the self-matching module, only the query's own character sequence is used to learn a good query semantic representation; then, the character-level matching module performs query and category matching in the character vector space to capture fine-grained matching features between the two; the semantic-level matching module measures the semantic similarity between the query and category representations to capture coarse-grained matching features; finally, these three features are connected and provided to the classifier to predict the classification result of the query information.

[0137] For example, a multi-granularity matched attention network can perform the following steps.

[0138] In step S1, vector representations of query information and object categories are extracted. First, vector representations of query information and each object category are extracted as query features and category features, respectively. Vector representation of query information and each object category is the basis for aligning the two into the same semantic space. A pre-trained language model, such as BERT, can be selected as an encoder for query information and each object category.

[0139] For example, for long-tail user queries, there is often a lack of sufficient training samples to accurately predict the user's intention. In this case, auxiliary knowledge such as the name information of the category and the core item words of the object (i.e., the keywords in the description information) can be used to assist the model in making classification decisions.

[0140] For example, in order to better learn the semantics of an object category, the character sequence of the object category consists of two parts: the name information of the object category n j represents the jth object category, L n Represents the total number of object categories; the core product words of each object contained in the jth object category L m Indicates the number of core product words.

[0141] In some embodiments, after obtaining the core product word Then, they are combined with the name information of the object category to which they belong [n j ] are connected to generate the category representation Will Input the feature extraction module BERT of the BERT model token (), encode the representation of the object category to extract the category features of the object category.

[0142] For example, in order to map query information and object categories into the same semantic space, query information and object categories can share the same BERT model.

[0143] For example, the query feature Q of query information i i and the category feature C of object category j j It can be expressed as follows:

[0144]

[0145]

[0146] BERT token () can be the last layer of the BERT model with the classification layer CLS removed; The token embedding matrices that can represent query information and object categories respectively.

[0147] In step S2, the features of the pure query text are extracted. The classification processing of the text classification model can be based on the pure query text, so the advantages of the self-attention mechanism can be used to extract the text features of the pure user query information, thereby improving the accuracy of classification.

[0148] For example, the query terms contained in the query information contribute differently to the entire query sentence vector, and an attention mechanism can be applied to summarize the query embedding matrix to extract intent-related query terms that are important for representing the query.

[0149] For example, we can use the query feature Q i , extract the self-attention feature q of the query information i :

[0150]

[0151]

[0152] These are the randomly initialized parameter vector and parameter matrix, both of which are parameters that can be learned through machine learning model training. i,t refers to the matrix u i In the query information, the word x t The corresponding vector; Q i,t refers to the matrix Q i In the query information, the word x t The corresponding vector.

[0153] For example, α = softmax(u i,t ) is an evaluation function used to evaluate the importance of the query word for the sentence representation of the current query information. i Feeding into the self-attention module, we can obtain self-attention features using only our own information is used to predict the user's intention.

[0154] In step S3, the fine-grained interactive features of query information-object category are extracted. For long-tail queries, when the model lacks sufficient training samples to accurately predict the user's intention, auxiliary knowledge such as the name of the object category and core product words are needed to assist the model in determining the classification decision. Therefore, the classification accuracy can be improved by extracting the first matching feature of the query information-object category as a fine-grained interactive feature.

[0155] For example, in e-commerce applications, the query information entered by users may contain typos or colloquial expressions, which are far from the official product titles and names of object categories. Therefore, the above embodiment learns the fine-grained similarity between query information and object categories at the character level, which is very important for the model to generalize grammatical queries and can improve classification accuracy.

[0156] For example, in order to extract fine-grained interaction features between query information and all object categories, we can i and category representation C j Then, we stack each query-category interaction matrix M on the channel dimension. j , which can be expressed as:

[0157]

[0158] M=[M1,M2,…,M j ,…,M Class ]

[0159] is a trainable weight parameter, is the interaction matrix between queries and categories, It is a feature map of character-level interactions between query information and each object category, and Class is the total number of object categories.

[0160] For example, in order to extract fine-grained matching features of each channel from the feature map M, a window size of r can be used. w × h The two-dimensional convolution module performs convolution operation on each channel of the feature map M as follows:

[0161]

[0162] σ(·) can be a ReLU activation function, k represents the kth channel of the feature map M, and W a,b is the convolution kernel, b is the bias vector, M i+a,j+b It is the i+a,j+bth element of the kth channel of the feature map.

[0163] For example, a 2D max pooling layer can be applied to the convolutional feature map Extract most of the important features i,j as follows:

[0164] p w and p h are the width and height of the 2D max pooling.

[0165] For example, the convolution feature map after the maximum pooling can be flattened (i.e., the matrix can be flattened to one dimension) and mapped to a low-dimensional space through a linear transformation layer, represented as the first matching feature The first matching feature contains the fine-grained matching features between the query information and each object category.

[0166] In step S4, semantic-level interactive matching features of query information and object categories are extracted. Because the words in the query information may be polysemous, relying solely on character-level matching may not be sufficient to fully capture the user's true intention. For example, "apple watch" and "apple juice" are highly correlated with "apple" at the character level, but they are different from each other at the semantic level. Therefore, in addition to considering the fine-grained matching features between query information and object types, the context-aware similarity between query information and object types, i.e., coarse-grained matching features, is also captured at the semantic level to improve classification accuracy.

[0167] For example, in order to obtain a category representation at the semantic level, the representation of the object category C j The time steps of the character sequence are mean pooled, and the representations of each mean-pooled object category are stacked together to determine the comprehensive representation of all object categories.

[0168] c j =mean(C j )

[0169]

[0170] For example, a criss-cross attention layer is applied to combine the query information with the comprehensive representation of all object categories to determine the second matching feature Z2:

[0171]

[0172] are trainable weights, It is a coarse-grained matching feature between query and category at the semantic level.

[0173] In step S5, the model is trained and inferred. The query representation q is obtained above i, both fine-grained query information-object category matching features Z1 and coarse-grained matching features Z2 help predict the user's intention, so they can be fused together using matrix multiplication.

[0174] For example, a nonlinear transformation layer can be introduced to i , Z1 and Z2 are fused to obtain the fused feature X:

[0175] X = ReLU(q i W qf +[Z1;Z2]W z )

[0176] are learnable weights.

[0177] In some embodiments, the prediction parameters of the query information for each object category can be determined based on the fusion features. As a classification result. For example, is a learnable parameter, Represents the prediction parameters of the query information for the object category c.

[0178] For example, you can use To represent the true classification label of the query information, y c ∈{0,1} indicates whether the query information belongs to the object category c. A multi-label cross entropy loss function can be used Train the machine learning model:

[0179]

[0180] σ(·) can be a sigmoid activation function.

[0181] In step S6, the model is deployed online and performs prediction services. Figure 3 The embodiments in the present invention implement online services.

[0182] Figure 3 Schematic diagram showing some embodiments of the online service architecture of the present disclosure.

[0183] like Figure 3 As shown in the figure, at the beginning of the workflow, the system maintains a local cache module to directly store the object category prediction results of high-frequency user queries in memory; if the query does not hit the cache, the online prediction process will be triggered. Three models can be used to implement the online prediction process: a first machine learning model containing a second classification module for intent prediction of long-tail queries; a first machine learning model containing a first classification module for object category prediction of long-tail queries; a second machine learning model for general query prediction (i.e., predicting item categories), such as the BERT model.

[0184] For example, in order to reduce the response latency of online deployment, a 4-layer BERT model can be extracted from the 12-layer BERT-base as a text encoder (feature extraction module), and the rest of the modules of the 4-layer BERT model remain unchanged.

[0185] In some embodiments, for the application scenario of long-tail queries, a first machine learning model is first used to determine whether a query information belongs to a specified scenario (such as whether it involves an object of a service category); then, if it does, the first machine learning model is used to predict the service object category corresponding to the query information in the scenario; if it does not, the second machine learning model is used to predict the item object category corresponding to the query information.

[0186] For example, the scene categories (such as service object categories) can be merged with the prediction results of ordinary BERT serving general queries (such as item object categories) into the final classification results to improve the classification accuracy.

[0187] Figure 4 A block diagram showing some embodiments of the information classification device of the present disclosure.

[0188] like Figure 4 As shown, the information classification device 4 includes: a convolution module 41, used to extract the first matching features of the user's query information and each object category in a plurality of object categories; a cross-attention module 42, used to extract the second matching features of the query information and each object category; a first classification module 43, used to determine the object category corresponding to the query information based on the first matching features and the second matching features.

[0189] In some embodiments, the convolution module 41 extracts the first matching feature according to the dot product processing result of the query feature of the query information and the category feature of each object category.

[0190] In some embodiments, the convolution module 41 calculates the dot product processing result based on the query information, the transposition of the category features and the weight parameters, and the weight parameters are determined by training the first machine learning model.

[0191] In some embodiments, the convolution module 41 performs two-dimensional convolution processing on the dot product processing results on multiple channels to determine a convolution feature map; and extracts the first matching feature based on the convolution feature map.

[0192] In some embodiments, the convolution module 41 performs maximum pooling processing on the convolution feature map to extract the first matching feature.

[0193] In some embodiments, the cross-attention module 42 performs mean pooling processing on the category features of each object category at the time step of the relevant information of each object category to determine the mean pooling processing result; and extracts the second matching feature based on the mean pooling processing result and the query feature of the query information.

[0194] In some embodiments, the information classification device 4 also includes: a feature extraction module 44, which is used to extract the category features of each object category based on the relevant information of each object category and the relevant information of the objects belonging to the object category, and extract the query features of the query information based on at least one word contained in the query information; the convolution module 41 extracts the first matching feature based on the query feature and the category feature; the cross-attention module 42 extracts the second matching feature based on the query feature and the category feature.

[0195] In some embodiments, the relevant information of the object category includes name information of the object category, and the relevant information of the object includes keywords in the description information of the object.

[0196] In some embodiments, the information classification device 4 also includes: a feature extraction module 44, which is used to extract query features of the query information based on at least one word contained in the query information; a self-attention module 45, which extracts self-attention features of the query information based on the query features; and a first classification module 43 determines the object category corresponding to the query information based on the self-attention features.

[0197] In some embodiments, the first classification module 43 determines the object category corresponding to the query information by using nonlinear transformation processing based on the first matching feature, the second matching feature and the self-attention feature.

[0198] In some embodiments, the object category includes at least one of an item category or a service category. When the query information involves an object of the service category, the first classification module determines the service category corresponding to the query information based on the first matching feature and the second matching feature.

[0199] In some embodiments, the information classification device 4 includes a first machine learning model, which includes a convolution module 41, a cross-attention module 42, and a first classification module 43.

[0200] In some embodiments, the information classification device 4 also includes: a second machine learning model 47, which is used to determine the item category corresponding to the query information based on the query characteristics of the query information and the category characteristics of each object category, and the second machine learning model 47 is different from the first machine learning model; in the case where the query information involves objects of the service category, the classification result of the query information is determined based on the service category corresponding to the query information and the item category corresponding to the query information.

[0201] In some embodiments, the information classification device 4 further includes: a second classification module 46, which is used to determine whether the query information involves an object of a service category.

[0202] In some embodiments, the first machine learning model is trained by a multi-label cross entropy loss function, which is calculated based on the object category corresponding to the query information sample and the annotation results of the query information sample.

[0203] Figure 5 A block diagram showing some other embodiments of the information classification device disclosed in the present invention.

[0204] like Figure 5 As shown, the information classification device 5 of this embodiment includes: a memory 51 and a processor 52 coupled to the memory 51 , and the processor 52 is configured to execute the information classification method in any one of the embodiments of the present disclosure based on the instructions stored in the memory 51 .

[0205] The memory 51 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, a database, and other programs.

[0206] Figure 6 The following is a block diagram showing some further embodiments of the information classification device disclosed herein.

[0207] like Figure 6 As shown, the information classification device 6 of this embodiment includes: a memory 610 and a processor 620 coupled to the memory 610 , and the processor 620 is configured to execute the information classification method in any of the aforementioned embodiments based on the instructions stored in the memory 610 .

[0208] The memory 610 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0209] The information classification device 6 may also include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650, the memory 610, and the processor 620 may be connected, for example, via a bus 660. The input / output interface 630 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, a touch screen, a microphone, and a speaker. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as an SD card and a USB flash drive.

[0210] In some embodiments, the present disclosure provides a server, comprising the information classification device in any one of the above embodiments.

[0211] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transient storage media including but not limited to disk storage, CD-ROM, optical storage, etc., which contain computer-usable program code.

[0212] So far, the information classification method, information classification device, server and non-volatile computer-readable storage medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed herein.

[0213] The method and system of the present disclosure may be implemented in many ways. For example, the method and system of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0214] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. An information classification method, comprising: Extracting the first matching feature between the user's query information and each of the plurality of object categories by using convolution processing; Extracting second matching features of the query information and each of the object categories using cross attention processing; An object category corresponding to the query information is determined according to the first matching feature and the second matching feature.

2. The information classification method according to claim 1, wherein: The extracting of the query information and the first matching feature of each object category in the plurality of object categories comprises: The first matching feature is extracted according to a dot product processing result of the query feature of the query information and the category feature of each object category.

3. The information classification method according to claim 2, wherein: The extracting the first matching feature comprises: The dot product processing result is calculated according to the query information, the transposition of the category feature and a weight parameter, wherein the weight parameter is determined by training the first machine learning model.

4. The information classification method according to claim 2, wherein: The extracting the first matching feature comprises: Performing two-dimensional convolution processing on the dot product processing results on multiple channels to determine a convolution feature map; The first matching feature is extracted according to the convolution feature map.

5. The information classification method according to claim 4, wherein: The extracting the first matching feature comprises: Perform maximum pooling processing on the convolutional feature map to extract the first matching feature.

6. The information classification method according to claim 1, wherein: The extracting the query information and the second matching feature of each object category comprises: At the time step of the relevant information of each object category, performing mean pooling processing on the category features of each object category to determine a mean pooling processing result; The second matching feature is extracted according to the mean pooling processing result and the query feature of the query information.

7. The information classification method according to claim 1, wherein: The step of extracting the first matching feature between the user's query information and each of the plurality of object categories comprises: Extracting a category feature of each object category according to the relevant information of each object category and the relevant information of objects belonging to the object category; extracting a query feature of the query information according to at least one word included in the query information; Extracting the first matching feature according to the query feature and the category feature; The extracting the query information and the second matching feature of each object category comprises: The second matching feature is extracted according to the query feature and the category feature.

8. The information classification method according to claim 7, wherein: The relevant information of the object category includes the name information of the object category, and the relevant information of the object includes the keywords in the description information of the object.

9. The information classification method according to claim 1, further comprising: extracting a query feature of the query information according to at least one word included in the query information; According to the query feature, extracting the self-attention feature of the query information by using self-attention processing; Wherein, determining the object category corresponding to the query information includes: According to the self-attention feature, determine the object category corresponding to the query information.

10. The information classification method according to claim 9, wherein: Determining the object category corresponding to the query information includes: According to the first matching feature, the second matching feature and the self-attention feature, a nonlinear transformation process is utilized to determine an object category corresponding to the query information.

11. The information classification method according to claim 1, wherein: The object category includes at least one of an item category or a service category, Determining the object category corresponding to the query information includes: In a case where the query information involves an object of a service category, the service category corresponding to the query information is determined based on the first matching feature and the second matching feature using a first classification module of a first machine learning model.

12. The information classification method according to claim 11, further comprising: determining, according to the query features of the query information and the category features of each object category, a category of items corresponding to the query information using a second machine learning model, the second machine learning model being different from the first machine learning model; In the case where the query information involves an object of a service category, a classification result of the query information is determined according to the service category corresponding to the query information and the item category corresponding to the query information.

13. The information classification method according to claim 11, further comprising: The second classification module of the first machine learning model is used to determine whether the query information involves an object of a service category.

14. The information classification method according to any one of claims 1 to 13, wherein: The first matching feature is extracted by a convolution module of a first machine learning model, the second matching feature is extracted by a cross-attention module of the first machine learning model, and the object category corresponding to the query information is determined by a first classification module of the first machine learning model, The first machine learning model is trained by a multi-label cross entropy loss function, and the multi-label cross entropy loss function is calculated according to the object category corresponding to the query information sample and the labeling result of the query information sample.

15. An information classification device, comprising: A convolution module, used to extract a first matching feature between the user's query information and each of the plurality of object categories; a cross-attention module for extracting a second matching feature between the query information and each of the object categories; The first classification module is used to determine the object category corresponding to the query information according to the first matching feature and the second matching feature.

16. An information classification device, comprising: Memory; and A processor coupled to the memory, wherein the processor is configured to execute the information classification method according to any one of claims 1 to 14 based on instructions stored in the memory.

17. A non-volatile computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the information classification method according to any one of claims 1 to 14 is implemented.

Citation Information

Patent Citations

  • Object classification method and device

    CN115410000A

  • Target detection method and device, equipment and storage medium

    CN115953665A

  • Method for re-recognizing object image based on multi-feature information capture and correlation analysis

    US20220415027A1