A multi-round dialogue product classification method and device and a storage medium

Abstract

The application discloses a kind of multi-wheel dialogue product classification method, device and storage medium, method includes: according to the input text in long multi-wheel dialogue text;Modeling is carried out in combination with character vector, fragment vector and hierarchical position coding vector, and sequence feature vector is extracted;Each sentence in sequence feature vector is input into model, and the vector output after the sentence is used as the sentence vector of the sentence;Attention weight of user product information to sentence vector is obtained by attention mechanism, and sentence vector is weighted and fused to obtain long text feature vector;Long text feature vector is spliced with user product information, and used as text representation feature vector;Product classification is carried out according to text representation feature vector, and final product classification result is obtained.The application converts absolute position coding into relative position coding by layering, and the performance of product classification is improved by fusing and representing sentence vector through attention mechanism, which can be widely applied in natural language processing field.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a multi-round dialogue product classification method, device and storage medium. Background Art

[0002] Text classification is one of the most fundamental and important tasks in natural language processing. It is a data mining technique based on text sequences. Early text classification tasks were mostly limited to rule-based or statistical learning methods such as keyword matching. In recent years, with the rapid rise of pre-trained language models such as BERT, RoBERTa, XLNet, and Ernie, academia and industry have begun to pre-train these models using massive amounts of internet text data to learn general language representations and prior knowledge. Fine-tuning is then used to further acquire task-specific knowledge. Numerous studies have confirmed the effectiveness of this two-stage pre-training + fine-tuning approach.

[0003] For the multi-turn long conversation text product classification task described in this article, there are numerous dialogue sentences, and not every sentence contains the same semantic information relevant to product classification. Furthermore, due to the limitations of absolute encoding, the mainstream BERT model can only process a maximum of 512 tokens. However, the length of the multi-turn long conversation text X almost always exceeds this limit. Using truncation to forcibly retain the input to 512 would significantly lose the original information. Currently, there is no technical solution to this problem. Summary of the Invention

[0004] In order to at least solve one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a multi-round dialogue product classification method, device and storage medium based on hierarchical position coding and sentence vector fusion.

[0005] The technical solution adopted in the present invention is:

[0006] A multi-round dialogue product classification method includes the following steps:

[0007] Obtain multi-user conversation recordings and convert them into multi-round long conversation texts;

[0008] Insert [CLS] and [SEP] tags before and after each sentence in the multi-turn long conversation text, and combine it with each user's product order information table Product to form the input text X;

[0009] Combine character vectors, segment vectors, and hierarchical positional encoding vectors designed for multi-round long conversation texts to build a model, and extract sequence feature vectors through the encoder, denoted as Encoder(X);

[0010] Each sentence in Encoder(X) is input into the model and the output vector is used as the sentence vector CLS of the sentence. text ;

[0011] Obtain user product information CLS through attention mechanism Product For sentence vector CLS text Attention weight, sentence vector CLS text Perform weighted fusion to obtain the long text feature vector CLS seq ;

[0012] The obtained long text feature vector CLS seq CLS with user product information Product Concatenate them into a text representation feature vector for multi-round long conversation texts;

[0013] Products are classified according to the text representation feature vector, and the final product classification result and loss value are obtained.

[0014] Furthermore, the model is trained in an unsupervised manner, wherein the training data is labeled in the following manner:

[0015] Extract keywords from each product's name and description text, and integrate keywords from multiple products to create a product keyword table;

[0016] Combined with multiple rounds of conversation text, the TF-IDF algorithm is used to assign weights to each keyword in the keyword table. Based on the keyword table, features are extracted for each product and call text to establish a relationship between the call text and the product.

[0017] Through similarity calculation and sorting, the product labels corresponding to the multi-round conversation texts are obtained.

[0018] Furthermore, the multi-round dialogue product classification method further includes the step of pre-processing the multi-round long dialogue text:

[0019] Each sentence in the multi-round conversation text is segmented and useless words are filtered out. Statements irrelevant to product classification are removed. The multi-round long conversation text Text = {text1, ..., textk, ..., textn} is reconstructed according to the conversation roles and conversation order. At the same time, the user's current product order information is filtered.

[0020] Furthermore, the expression of the input text X is:

[0021] X=[CLS]Product[SEP][CLS]text1[SEP][CLS]...[SEP][CLS]text n [SEP]

[0022] Furthermore, the model is constructed by combining character vectors, segment vectors, and hierarchical position encoding vectors designed for multi-round long dialogue texts, and extracting sequence feature vectors through an encoder, which is denoted as Encoder(X), including:

[0023] Using the absolute position encoding vectors p1, p2, ..., p that BERT has trained n , by hierarchically constructing a new set of relative encoding vectors q1, q2, ..., q m :

[0024] q (i-1)×n+j =αp i +(1-α)p j

[0025] Wherein, n and m are both positive integers, and n<m; α∈(0,1), and α≠0.5;

[0026] The input text X is modeled by combining character vectors, fragment vectors, and hierarchical position encoding vectors designed for multi-round long dialogue texts, and the sequence feature vector Encoder(X) is obtained through the encoder.

[0027] Furthermore, the user product information CLS is obtained through the attention mechanism Product For sentence vector CLS text Attention weight, sentence vector CLS text Perform weighted fusion to obtain the long text feature vector CLS seq ,include:

[0028] Use attention mechanism to classify sentence vectors CLS text Perform weight evaluation to obtain sentence vector CLS text The attention weight of

[0029] Get user product information CLS Product For each sentence vector CLS text Attention weight, according to the attention weight, all sentence vectors CLS text Perform weighted fusion to obtain a long text feature vector CLS with more significant product classification features seq :

[0030] CLS seq =∑a i CLS texti

[0031] Among them, a i is the attention weight corresponding to the i-th sentence vector, CLS texti is the i-th sentence vector.

[0032] Furthermore, the obtained long text feature vector CLS seq CLS with user product information Product Concatenate them as the text representation feature vector of multi-round long conversation text, including:

[0033] The long text feature vector CLS seq CLS with user product information Product To splice:

[0034] H cls =(CLS Product , CLS seq )

[0035] Get the text representation feature vector H cls .

[0036] Furthermore, the product classification is performed according to the text representation feature vector to obtain the final product classification result and the loss value, including:

[0037] The text is represented by the feature vector H cls The fully connected linear classification layer FNN of the input model is used to perform product classification tasks. After mapping, the probability distribution for product classification is obtained, which is expressed as P(Y|X):

[0038] P(Y|X)=Softmax(W·H cls +b)

[0039] Where W and b are both trainable parameters;

[0040] Get the product category corresponding to the maximum value of the probability distribution P(Y|X) as the final prediction result y. The calculation process is as follows:

[0041]

[0042] Where N is the total number of product classification labels; P(y i |X) represents the probability of the text in the i-th product label;

[0043] According to the cross entropy loss function, the distance between the probability distribution P(Y|X) and the target distribution is calculated to obtain the loss value Loss:

[0044] Loss=-log P(y i |X)

[0045] Furthermore, the training step of the model includes:

[0046] Using the pre-trained BERT language model, the product classification module is fine-tuned based on the BERT language model; the fine-tuning process uses a batch training method, and the initial learning rate is set to 5e -5 Default value; during training, the Adamw optimizer with learning rate ramp-up and exponential decay is used for optimization.

[0047] Another technical solution adopted in the present invention is:

[0048] A multi-round dialogue product classification device, comprising:

[0049] at least one processor;

[0050] at least one memory for storing at least one program;

[0051] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0052] Another technical solution adopted in the present invention is:

[0053] A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to perform the method described above when executed by the processor.

[0054] The present invention improves product classification performance by converting absolute position encodings into relative position encodings through a layered approach and fusing sentence vectors using an attention mechanism. It can fully process multi-turn, long conversations exceeding the original BERT input length limit and fully understand text semantics through user order information, achieving better results in product classification tasks in complex contexts. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0056] Figure 1 1 is a flow chart of a multi-round dialogue product classification method based on hierarchical position coding and sentence vector fusion in an embodiment of the present invention;

[0057] Figure 2 This is a flowchart of the steps of a multi-round dialogue product classification method based on hierarchical position coding and sentence vector fusion in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0059] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention.

[0060] In the description of the present invention, the meaning of "several" is one or more, the meaning of "many" is two or more, and "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of the first and the second, it is only for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, "and / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship.

[0061] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0062] like Figure 1 and Figure 2 As shown, this embodiment provides a multi-round dialogue product classification method based on hierarchical position encoding and sentence vector fusion, the method comprising the following steps:

[0063] S1. Obtain multi-user conversation recordings and convert the multi-user conversation recordings into multi-round long conversation texts.

[0064] Convert multi-user conversation recordings into multi-round long conversation texts, perform word segmentation and useless word filtering on each sentence, remove sentences that are not helpful for product classification, and then reconstruct the multi-round long conversation text according to the conversation roles and conversation order. k ,…,text n}, the process aims to eliminate irrelevant information and shorten the length of sentences, thereby reducing the training time and resource consumption of the model.

[0065] In some embodiments, for a given multi-turn conversation text dataset, the multi-turn conversation speech is converted into text format. To improve the efficiency of subsequent data processing, each sentence in the multi-turn conversation text is segmented and useless words are filtered. Sentences that are unhelpful for product classification are removed, and the multi-turn long conversation text Text = {text1,…,textk,…,textn} is reconstructed based on the conversation roles and conversation sequence. The user's current product order information is also preprocessed in the same way.

[0066] S2. Insert [CLS] and [SEP] tags before and after each sentence in the multi-round long conversation text, and combine it with each user's product order information table Product to form the input text X.

[0067] Specifically, [CLS] and [SEP] tags are inserted before and after each sentence of the multi-round long dialogue text, and combined with each user's product order information table Product, the input text X = [CLS]Product[SEP][CLS]text1[SEP][CLS]…[SEP][CLS]text n [SEP].

[0068] S3. Combine character vectors, fragment vectors, and hierarchical position encoding vectors designed for multi-round long dialogue texts to build a model, and extract sequence feature vectors through the encoder, denoted as Encoder(X).

[0069] Since the mainstream BERT model can process a maximum of 512 tokens, the fundamental reason is that BERT's position encoding method is absolute position encoding, which is set to a maximum of 512. However, the length of the multi-round long dialogue text X almost exceeds this limit. If the input is forcibly retained to 512 by truncation, a large amount of original information will be lost. Therefore, this embodiment uses the absolute position encoding vectors p1, p2, ..., p n By constructing a new set of relative encoding vectors q1,q2,…,q m (m>n).

[0070] q (i-1)×n+k =αp i+(1-α)p j (1)

[0071] Where α∈(0,1) and α≠0.5. The purpose of this is to represent (i-1)×n+j hierarchically as (i, j), and then the position codes corresponding to i and j are αp respectively. i and (1-α)p j , and the final relative encoding vector is the superposition of the two absolute encodings. In this way, the encoding parameters of the model are still the original BERT trained p1, p2, ..., p n , but in fact it can be expressed as n 2 A relative position encoding is used to meet the model's processing requirements for multi-round long dialogue texts.

[0072] The input text X is modeled by combining character vectors, segment vectors, and hierarchical position encoding vectors designed for multi-round long dialogue texts, and the sequence feature vector Encoder(X) is obtained through the encoder. The output vector CLS of user product information is extracted from it. Product And the sentence vector CLS of the clauses of multi-round long dialogues text .

[0073] S4. Each sentence in Encoder (X) is input into the model and the output vector is used as the sentence vector CLS of the sentence. text .

[0074] S5. Obtain user product information CLS through attention mechanism Product For sentence vector CLS text Attention weight, sentence vector CLS text Perform weighted fusion to obtain the long text feature vector CLS seq .

[0075] In multi-turn long conversations, not every sentence contains the same semantic information relevant to product classification. To assess the importance of user-ordered product information for the product classification task, we use an attention mechanism to weight sentence vectors:

[0076]

[0077] Where a is the sentence vector attention weight, which represents the attention weight of each short sentence in a multi-round long conversation text. Q represents the query vector Query when performing attention calculation. K represents the vector matrix Key to which different attention weights need to be assigned. d is a normalization parameter that is independent of the specific task content.

[0078] To obtain the attention weight of each sentence vector of the user's ordered product information, Q takes CLSProduct , K takes the sequence CLS composed of the sentence vectors of all clauses text-seq Calculation can be performed to obtain CLS Product For each sentence vector CLS text The attention weight a is then used to perform weighted fusion on all sentence vectors according to the weights to obtain a sentence vector fusion feature CLS with more significant classification features. seq :

[0079] CLS seq =∑a i CLS texti (3)

[0080] S6. Obtain the long text feature vector CLS seq CLS with user product information Product The concatenation is performed as the text representation feature vector of the multi-round long conversation text.

[0081] The sentence vector obtained above is fused with the feature CLS seq Vector CLS of product information ordered by user Product To splice:

[0082] H cls =(CLS Product , CLS seq ) (4)

[0083] S7. Classify the products according to the text representation feature vector, obtain the final product classification result, and obtain the loss value.

[0084] H CLS The fully connected linear classification layer FNN of the product classification module is input to perform the product classification task. After mapping, the probability distribution for product classification is obtained, which is expressed as P(Y|X):

[0085] P(Y|X)=Softmax(W·H cls +b) (5)

[0086] Where W∈R n×d , R n×d Represents a two-dimensional real matrix, b∈R n , and W and b are both trainable parameters.

[0087] Get the product category corresponding to the maximum value of P(Y|X) as the final prediction result y. The calculation process is as follows:

[0088] y = argmax 0≤i≤N P(y i |X) (6)

[0089] Where N is the total number of product classification labels; P(y i |X) represents the probability of the text in the i-th product label;

[0090] Finally, according to the cross entropy loss function, the distance between the probability distribution P(Y|X) and the target distribution is calculated to obtain the loss value Loss:

[0091] Loss=-log P(y i |X) (7)

[0092] As an optional implementation, the model is trained in the following manner:

[0093] We use the pre-trained XLnet language model, which has rich semantic information learned from massive amounts of text, to fine-tune the product classification module. The fine-tuning process uses a batch training method with an initial learning rate of 5e -5 The default value is 5 epochs. In addition, the Adamw optimizer with learning rate ramping and exponential decay is used for optimization during training.

[0094] As an optional implementation, the model training phase is proposed. Due to the lack of labeled training data and the high cost of manual labeling, which cannot meet actual business needs, an unsupervised labeling method based on keywords and weights is proposed based on the characteristic words in the product names. The idea of ​​this method is as follows:

[0095] a) Extract keywords from each product name and description text, and integrate the keywords of 739 products to create a product keyword table.

[0096] b) Combining multiple rounds of conversation text, use the TF-IDF algorithm to assign weights to each keyword in the keyword list. Based on the keyword list, extract features for each product and call text to establish a relationship between the call text and the product.

[0097] c) Obtain labels corresponding to multiple rounds of dialogue texts through similarity calculation and sorting.

[0098] In summary, the embodiment of the present invention uses a given multi-round long dialogue text dataset to design a multi-round dialogue product classification method based on hierarchical position encoding and sentence vector fusion, thereby improving the accuracy of the model in product classification.

[0099] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0100] (1) The multi-round dialogue product classification method based on hierarchical position coding and sentence vector fusion provided by the application can convert absolute position coding trained by BERT into relative position coding through hierarchical conversion on the basis of the absolute position coding trained by BERT, break through the limitation that the input length of the BERT model is limited to 512, and realize full utilization of multi-round long dialogue text.

[0101] (2) The product classification module provided by the application realizes full utilization of user product ordering list information by fusing and representing sentence vectors through an attention mechanism.

[0102] (3) The application designs a data labeling algorithm based on keywords and weights to assist in labeling the relationship between the dialogue and the product in order to avoid the huge cost caused by manual data labeling, and reduces the workload of manual labeling.

[0103] The embodiment also provides a multi-round dialogue product classification device, which comprises:

[0104] at least one processor;

[0105] at least one memory for storing at least one program;

[0106] When the at least one program is executed by the at least one processor, the at least one processor implements the method. Figure 2 The method.

[0107] The multi-round dialogue product classification device of the embodiment can execute the multi-round dialogue product classification method provided by the method embodiment, can execute any combination of the method embodiment, has the corresponding functions and advantages of the method.

[0108] The embodiment of the application also discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method. Figure 2 The method.

[0109] The embodiment also provides a storage medium storing instructions or programs that can execute the multi-round dialogue product classification method provided by the method embodiment, and when the instructions or programs are executed, any combination of the method embodiment can be executed, and the corresponding functions and advantages of the method are possessed.

[0110] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0111] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0112] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0113] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0114] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0115] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0116] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0117] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0118] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A multi-round dialogue product classification method, characterized in that: It includes the following steps: Obtain multi-user conversation recordings and convert the multi-user conversation recordings into multi-turn long conversation texts; Insert [CLS] and [SEP] tags before and after each sentence in the multi-turn long conversation text, and combine each user's product order information table Product to form the input text X; Model by combining character vectors, segment vectors, and hierarchical position encoding vectors designed for multi-turn long conversation texts, and extract sequence feature vectors through an encoder, denoted as Encoder(X); Each sentence in Encoder(X) is input into the model and the output vector is used as the sentence vector CLS of the sentence. text ; Obtain user product information CLS through attention mechanism Product For sentence vector CLS text Attention weight, sentence vector CLS text Perform weighted fusion to obtain the long text feature vector CLS seq ; The obtained long text feature vector CLS seq CLS with user product information Product Concatenate them into a text representation feature vector for multi-round long conversation texts; Classify products according to the text representation feature vectors, obtain the final product classification result, and obtain the loss value; The modeling by combining character vectors, segment vectors, and hierarchical position encoding vectors designed for multi-turn long conversation texts, and extracting sequence feature vectors through an encoder, denoted as Encoder(X), includes: Using the absolute position encoding vector p1, p2, ..., p that BERT has trained n , by constructing a new set of relative encoding vectors q1,q2,…,q m : q (i-1)×n+j =αp i +(1-α)p j Where n and m are both positive integers, and n < m; α ∈ (0, 1), and α ≠ 0.5; Model the input text X by combining character vectors, segment vectors, and hierarchical position encoding vectors designed for multi-turn long conversation texts, and obtain the sequence feature vector Encoder(X) through an encoder; The user product information CLS is obtained through the attention mechanism Product For sentence vector CLS text Attention weight, sentence vector CLS text Perform weighted fusion to obtain the long text feature vector CLS seq ,include: Use attention mechanism to classify sentence vectors CLS text Perform weight evaluation to obtain sentence vector CLS text The attention weight of Get user product information CLS Product For each sentence vector CLS text Attention weight, according to the attention weight, all sentence vectors CLS text Perform weighted fusion to obtain a long text feature vector CLS with more significant product classification features seq : CLS seq =∑a i CLS texti Among them, a i is the attention weight corresponding to the i-th sentence vector, CLS texti is the i-th sentence vector.

2. The multi-round dialogue product classification method according to claim 1, characterized in that: The model is trained in an unsupervised manner, and the training data is labeled in the following way: Extract keywords through the name and description text of each product, and integrate the keywords of multiple products to establish a product keyword table; Combine multi-turn conversation texts, use the TF-IDF algorithm to set weights for each keyword in the keyword table, and then extract features for each product and incoming call text according to the keyword table to establish the relationship between the incoming call text and the product; Obtain the product label corresponding to the multi-turn conversation text through similarity calculation and sorting.

3. The multi-round dialogue product classification method according to claim 1, characterized in that: The multi-turn conversation product classification method further includes a step of preprocessing the multi-turn long conversation text: Each sentence in the multi-round dialogue text is segmented and useless words are filtered, and sentences irrelevant to product classification are removed. The multi-round long dialogue text Text={text1,…,text k ,…,text s }; At the same time, filter the user's current product order information.

4. The multi-round dialogue product classification method according to claim 1, characterized in that: The long text feature vector CLS obtained seq CLS with user product information Product Concatenate them as the text representation feature vector of multi-round long conversation text, including: The long text feature vector CLS seq CLS with user product information Product To splice: H cls =(CLS Product ,CLS seq ) Get the text representation feature vector H cls .

5. The multi-round dialogue product classification method according to claim 1, characterized in that: The classifying products according to the text representation feature vectors, obtaining the final product classification result, and obtaining the loss value includes: Represent the text as a feature vector H cls The fully connected linear classification layer FNN of the input model is used to perform product classification tasks. After mapping, the probability distribution for product classification is obtained, which is expressed as P(Y|X): P(Y|X)=Softmax(W·H cls +b) In the formula, W and b are both trainable parameters; Obtain the product category corresponding to the maximum value of the probability distribution P(Y|X) as the final prediction result y, and the calculation process is as follows: Where N is the total number of product classification labels; P(y i |X) represents the probability of the text in the i-th product label; According to the cross-entropy loss function, calculate the distance between the probability distribution P(Y|X) and the target distribution to obtain the loss value Loss: Loss=-log P(y i |X)。 6. The multi-round dialogue product classification method according to claim 1, characterized in that: The training steps of the model include: Using the pre-trained BERT language model, the product classification module is fine-tuned based on the BERT language model; the fine-tuning process uses a batch training method, and the initial learning rate is set to 5e -5 Default value; during training, the Adamw optimizer with learning rate ramp-up and exponential decay is used for optimization.

7. A multi-round dialogue product classification device, characterized in that: Include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1-6.

8. A computer-readable storage medium storing a program executable by a processor, characterized in that: The program executable by the processor is used to execute the method according to any one of claims 1-6 when executed by the processor.

Images