Text category recognition method, device, terminal device and storage medium

Abstract

The embodiments of the present application relate to the field of financial technology, and specifically provide a method, apparatus, terminal device, and storage medium for identifying text categories. The method includes: splicing sample data according to a first splicing strategy to obtain a first spliced ​​text, and further splicing the first spliced ​​text according to a second splicing strategy to obtain a second spliced ​​text; using a language model to obtain first matrix information from the first spliced ​​text; using a language model to perform data prediction on the second spliced ​​text to obtain a distribution deviation evaluation index; inputting a query statement into the language model to obtain second matrix information; splicing the first matrix information and the second matrix information to obtain splicing matrix information; obtaining a query matrix corresponding to the query statement, and then determining the text category corresponding to the query statement based on the query matrix, the splicing matrix information, and the distribution deviation evaluation index, thereby improving the recognition accuracy of the text category, thereby improving the service quality, and thus improving business satisfaction.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to a method, apparatus, terminal device and storage medium for identifying text categories. Background Art

[0002] In the era of information explosion, manually processing and classifying large amounts of text data is a time-consuming and challenging task. With the development of artificial intelligence technology, people hope to use artificial intelligence methods to automate the text classification process to produce more reliable and less subjective results, thereby improving information retrieval efficiency and alleviating the problem of information overload.

[0003] In the existing technology, when performing text classification, due to the limitations of hardware equipment, there are strict restrictions on the input length when performing text classification. When the input length of the text is greater than the threshold, the text will be truncated, which will cause the truncated content to be lost during classification and recognition, reducing the accuracy of text classification and recognition. This also greatly restricts the reasoning ability of the model in the existing technology. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to provide a text category recognition method, apparatus, terminal device and storage medium, aiming to solve the problem that during the insurance recommendation process, longer speech information of the user is obtained, but when the speech information is subjected to text category recognition, when the speech information exceeds a threshold, the speech information will be truncated before text category recognition is performed, which reduces the accuracy of text category recognition, thereby reducing the accuracy of determining user needs and reducing the user experience.

[0005] In a first aspect, an embodiment of the present application provides a method for identifying a text category, comprising:

[0006] Obtaining sample data, the sample data including text data and a text category corresponding to the text data, and determining a first splicing strategy and a second splicing strategy for data splicing of the text data and the text category;

[0007] Splicing the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and further splicing the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text;

[0008] Using the first concatenated text using a language model, obtain first matrix information corresponding to the self-attention module of the first concatenated text in the language model;

[0009] Performing data prediction on the second concatenated text using the language model to obtain a prediction result, and then determining a distribution deviation evaluation index corresponding to the sample data based on the prediction result;

[0010] Obtaining a query statement to be predicted, inputting the query statement into the language model, and obtaining second matrix information corresponding to the query statement in the attention module of the language model;

[0011] Splicing the first matrix information and the second matrix information to obtain spliced ​​matrix information;

[0012] A query matrix corresponding to the query statement is obtained, and then a text category corresponding to the query statement is determined according to the query matrix, the concatenation matrix information, and the distribution deviation evaluation index.

[0013] In a second aspect, an embodiment of the present application further provides a text category recognition device, comprising:

[0014] A data acquisition module is used to obtain sample data, the sample data including text data and a text category corresponding to the text data, and determine a first splicing strategy and a second splicing strategy for data splicing of the text data and the text category;

[0015] a data splicing module, configured to splice the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and further splice the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text;

[0016] A data processing module, configured to use a language model to obtain first matrix information corresponding to a self-attention module of the first concatenated text in the language model;

[0017] a data prediction module, configured to perform data prediction on the second concatenated text using the language model to obtain a prediction result, and then determine a distribution deviation evaluation index corresponding to the sample data based on the prediction result;

[0018] A data query module, configured to obtain a query statement to be predicted, input the query statement into the language model, and obtain second matrix information corresponding to the query statement in the attention module of the language model;

[0019] a matrix splicing module, configured to splice the first matrix information and the second matrix information to obtain spliced ​​matrix information;

[0020] The data analysis module is used to obtain a query matrix corresponding to the query statement, and then determine the text category corresponding to the query statement based on the query matrix, the splicing matrix information and the distribution deviation evaluation index.

[0021] In a third aspect, an embodiment of the present application further provides a terminal device, the terminal including a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the steps of any text category recognition method provided in the specification of this application are implemented.

[0022] In a fourth aspect, an embodiment of the present application further provides a storage medium for computer-readable storage, characterized in that the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the text category recognition method as provided in any one of the items in the specification of this application.

[0023] An embodiment of the present application provides a method, apparatus, terminal device, and storage medium for identifying text categories, wherein the method includes obtaining example data, the example data including text data and a text category corresponding to the text data, and determining a first splicing strategy and a second splicing strategy for data splicing of the text data and the text category; performing data splicing on the example data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the example data, and continuing to splice the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text; using a language model to obtain first matrix information corresponding to a self-attention module of the first spliced ​​text in the language model from the first spliced ​​text; performing data prediction on the second spliced ​​text using the language model to obtain a prediction result, and then determining a distribution deviation evaluation index corresponding to the example data based on the prediction result; obtaining a query statement to be predicted, inputting the query statement into the language model, and obtaining second matrix information corresponding to the query statement in the attention module of the language model; splicing the first matrix information and the second matrix information to obtain splicing matrix information; obtaining a query matrix corresponding to the query statement, and then determining the text category corresponding to the query statement based on the query matrix, the splicing matrix information, and the distribution deviation evaluation index. This application uses multiple sets of sample data to obtain corresponding matrix information, and splices the matrix information to obtain spliced ​​matrix information, so that longer text content can be processed when performing text category recognition, thereby improving the accuracy of text category recognition. In the field of finance and insurance, after obtaining the content of the call between the customer and the salesperson, the text category classification can be effectively performed after the voice recognition of the call content, and then accurate words can be recommended to the salesperson to improve the salesperson's business level, thereby further improving the business transaction rate, or when the text category classification results are accurately obtained, content that is more in line with the user's needs can be recommended to the user, thereby improving the quality of user content recommendations, thereby improving the accuracy of user demand judgment and improving user experience and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0025] Figure 1 A flowchart of a text category recognition method provided in an embodiment of the present application;

[0026] Figure 2 for Figure 1 A flowchart of steps corresponding to a specific implementation of step S2;

[0027] Figure 3 for Figure 1 A flowchart of steps corresponding to a specific implementation of step S7;

[0028] Figure 4 A schematic diagram of the module structure of a text category recognition device provided in an embodiment of the present application;

[0029] Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0031] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0032] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0033] Embodiments of the present application provide a method, apparatus, terminal device, and storage medium for identifying text categories. The text category identification method can be applied to a terminal device, which can be a tablet computer, a laptop computer, a personal digital assistant, a wearable device, or a server, wherein the server can be a standalone server or a server cluster.

[0034] Among them, an embodiment of the present application provides a method, apparatus, terminal device and storage medium for identifying text categories, wherein the method includes obtaining example data, the example data including text data and a text category corresponding to the text data, and determining a first splicing strategy and a second splicing strategy for data splicing of the text data and the text category; performing data splicing on the example data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the example data, and continuing to splice the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text; using a language model to obtain the first matrix information corresponding to the self-attention module of the first spliced ​​text in the language model; using the language model to perform data prediction on the second spliced ​​text to obtain a prediction result, and then determining a distribution deviation evaluation index corresponding to the example data based on the prediction result; obtaining a query statement to be predicted, inputting the query statement into the language model, and obtaining the second matrix information corresponding to the query statement in the attention module of the language model; splicing the first matrix information and the second matrix information to obtain splicing matrix information; obtaining a query matrix corresponding to the query statement, and then determining the text category corresponding to the query statement based on the query matrix, the splicing matrix information and the distribution deviation evaluation index. This application uses multiple sets of sample data to obtain corresponding matrix information, and splices the matrix information to obtain spliced ​​matrix information, so that longer text content can be processed when performing text category recognition, thereby improving the accuracy of text category recognition. In the field of finance and insurance, after obtaining the content of the call between the customer and the salesperson, the text category classification can be effectively performed after the voice recognition of the call content, and then accurate words can be recommended to the salesperson to improve the salesperson's business level, thereby further improving the business transaction rate, or when the text category classification results are accurately obtained, content that is more in line with the user's needs can be recommended to the user, thereby improving the quality of user content recommendations, thereby improving the accuracy of user demand judgment and improving user experience and satisfaction.

[0035] The following embodiments of the present application are described in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0036] Please refer to Figure 1 , Figure 1 A flowchart of a text category recognition method provided in an embodiment of the present application.

[0037] like Figure 1As shown, the text category recognition method includes steps S1 to S7.

[0038] Step S1: obtaining sample data, the sample data including text data and a text category corresponding to the text data, and determining a first splicing strategy and a second splicing strategy for data splicing of the text data and the text category.

[0039] Exemplarily, multiple sets of example data are obtained, each set of example data including multiple pieces of text data and text categories corresponding to the text data. A strategy for splicing the multiple pieces of text data and text categories in each set of example data is determined, including a first splicing strategy and a second splicing strategy, wherein the first splicing strategy and the second splicing strategy may be the same or different.

[0040] For example, let's use text category recognition for emotion recognition. For example, if the text data is "I recently bought a car and would like to buy suitable insurance for it. Any recommendations?" and the text category is "car insurance," the first concatenation strategy could be "I recently bought a car and would like to buy suitable insurance for it. Any recommendations?" It is car insurance. In other words, the first concatenation strategy is "text data + it is + text category." Therefore, if the sample data includes multiple pieces of data, concatenating them according to the first concatenation strategy yields a concatenation result in the form: "text data 1 + it is + text category 1 + text data 2 + it is + text category 2 + ...."

[0041] In some embodiments, obtaining the sample data includes: determining sample data; and randomly grouping the sample data to obtain multiple groups of sample data.

[0042] For example, the identification of text categories includes multiple sample data, which are randomly divided into M groups, each of which is called example data, so that subsequent text category identification is achieved based on the multiple groups of example data.

[0043] For example, there are N sample data for sentiment analysis, which are randomly divided into M groups. Each group consists of multiple text data and text categories.

[0044] Step S2: performing data splicing on the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and further performing data splicing on the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text.

[0045] For example, after obtaining the sample data, the data in the sample data are sequentially spliced ​​according to the first splicing strategy to obtain the first spliced ​​text. After obtaining the first spliced ​​text, splicing is continued based on the first spliced ​​text according to the second splicing strategy to obtain the second spliced ​​text.

[0046] Please refer to Figure 2 In some embodiments, the example data is spliced ​​according to the first splicing strategy to obtain a first spliced ​​text corresponding to the example data, and the first spliced ​​text is further spliced ​​according to the second splicing strategy to obtain a second spliced ​​text. Step S2 includes steps S21 to S22.

[0047] Step S21: performing data splicing on the text data and the text category according to the first splicing strategy, thereby obtaining a first spliced ​​text corresponding to the sample data.

[0048] Exemplarily, the text data and text categories in the example data are spliced ​​in sequence according to the first splicing strategy, thereby obtaining a first spliced ​​text corresponding to the example data.

[0049] For example, the first concatenation strategy is: If the text data contains "I recently bought a car and would like to buy suitable insurance for it. Any recommendations?" and the text category is "Car Insurance," the first concatenation strategy could be "I recently bought a car and would like to buy suitable insurance for it. Any recommendations?" It is car insurance. Therefore, if the example data includes Text 1 in Category 1 and Text 2 in Category 2, the first concatenation text would be "Text 1. It is Category 1. Text 2. It is Category 2."

[0050] Step S22: determining a third spliced ​​text according to the second splicing strategy, and splicing the first spliced ​​text and the third spliced ​​text according to the second splicing strategy to obtain the second spliced ​​text.

[0051] For example, a third spliced ​​text that needs to be spliced ​​into the first spliced ​​text is determined according to the second splicing strategy. After obtaining the third spliced ​​text, the first spliced ​​text and the third spliced ​​text are spliced ​​according to the second splicing strategy to obtain a second spliced ​​text.

[0052] For example, the second concatenation strategy is that the third concatenated text is "NA. It is.", and the third concatenated text is added to the end of the first concatenated text. Then, the second concatenated text can be obtained as Text 1. It is category 1. Text 2. It is category 2. NA. It is."

[0053] Step S3: Using a language model to obtain the first matrix information corresponding to the self-attention module of the first concatenated text in the language model.

[0054] Exemplarily, the first concatenated text is input into the GPT-3 language model, thereby obtaining the first matrix information corresponding to the self-attention module of the first concatenated text in GPT-3.

[0055] Optionally, the language model can be GPT-3, chat GPT, or a language model that can simulate human natural language capabilities. This application does not impose any restrictions and you can choose one based on your needs.

[0056] In some embodiments, the first matrix information includes a first key matrix and a first value matrix, and the method of using the first concatenated text to obtain the first matrix information corresponding to the self-attention module of the first concatenated text in the language model includes: determining the text length corresponding to the first concatenated text, and when the text length is lower than the target text length, padding the first concatenated text to obtain target text data; and inputting the target text data into the language model to obtain the first key matrix and the first value matrix corresponding to the self-attention module of the target text data in the language model.

[0057] For example, after obtaining the first concatenated text corresponding to the sample data, all the first concatenated texts can be right-aligned so that they have the same maximum position index. Therefore, all the first concatenated texts can have the same relative distance to the test input. The text length corresponding to each first concatenated text is obtained. When the text length is less than the target text length, the left padding method can be used to add a left padding to the left of the input sequence. <pad>Mark, and then obtain the target text data, so as to input the target text data into the language model to obtain the first key matrix and the first value matrix corresponding to the self-attention module of the target text data in the language model.

[0058] Optionally, you can choose left alignment when aligning data, and you can set it according to your needs.

[0059] Step S4: performing data prediction on the second concatenated text using the language model to obtain a prediction result, and then determining a distribution deviation evaluation index corresponding to the example data according to the prediction result.

[0060] For example, the second concatenated text is input into the GPT-3 language model to obtain a prediction result corresponding to the second concatenated text, that is, a distribution result of the text categories corresponding to the second concatenated text. Based on the prediction result, the distribution result between the distributions is calculated, and the deviation between the example data and the uniform distribution is measured, thereby determining the distribution deviation evaluation index corresponding to the example data.

[0061] In some embodiments, the prediction result includes a prediction category and a prediction probability corresponding to the prediction category, and determining the distribution deviation evaluation index corresponding to the example data based on the prediction result includes: calculating the relative entropy of the prediction result based on the prediction category and the prediction probability to obtain a corresponding relative entropy result; and determining the distribution deviation evaluation index corresponding to the example data based on the relative entropy result.

[0062] Exemplarily, the prediction result includes all types of text category recognition and the prediction probabilities between the various types, and the sum of all the prediction probabilities is equal to 1.

[0063] Exemplarily, the relative entropy of the prediction result is calculated according to the prediction category and the prediction probability to obtain the corresponding relative entropy result, and the calculation method is shown in Formula 1.

[0064]

[0065] Among them, P(x) represents the corresponding predicted probability when the predicted category is x, and Q(x) represents the theoretical corresponding predicted probability when the predicted category is x, that is, Q(x) can be expressed as the corresponding predicted probability when the uniform distribution is used.

[0066] Step S5: Obtain a query statement to be predicted, input the query statement into the language model, and obtain second matrix information corresponding to the query statement in the attention module of the language model.

[0067] Exemplarily, the second matrix information includes a second key matrix and a second value matrix, obtains the query statement to be predicted, and inputs the query statement into the GPT-3 language model, thereby obtaining the second key matrix and second value matrix corresponding to the query statement in the attention module of the language model.

[0068] Step S6: Splicing the first matrix information and the second matrix information to obtain spliced ​​matrix information.

[0069] Exemplarily, the first matrix information includes a first key matrix and a first value matrix, and the second matrix information includes a second key matrix and a second value matrix. The first key matrix K1 corresponding to the example data and the second key matrix Kx corresponding to the query statement are obtained, and the first key matrix K1 and the second key matrix Kx are spliced ​​to obtain a spliced ​​key matrix [K1, Kx].

[0070] Optionally, when the sample data is multiple sets of data, multiple sets of key matrices can be obtained, namely the first key matrix K1, the second key matrix K2...the Nth key matrix KN, and the obtained concatenated key matrix is ​​[K1, K2,..., KN, Kx]. Among them, m a is the length of the text sequence after data splicing of the example data of group a, d is the hidden state dimension, m x is the sequence length corresponding to the query statement.

[0071] Exemplarily, a first value matrix V1 corresponding to the example data and a second value matrix Vx corresponding to the query statement are obtained, and then the first value matrix V1 and the second value matrix Vx are concatenated to obtain a concatenated value matrix [V1, Vx].

[0072] Optionally, when the sample data is multiple sets of data, multiple sets of value matrices can be obtained, namely the first value matrix V1, the second value matrix V2...the third value matrix VN, and the obtained spliced ​​value matrix is ​​[V1, V2,..., VN, Vx]. Among them, m a is the length of the text sequence after data splicing of the example data of group a, d is the hidden state dimension, m x is the sequence length corresponding to the query statement.

[0073] Step S7: Obtain a query matrix corresponding to the query statement, and then determine the text category corresponding to the query statement based on the query matrix, the concatenation matrix information, and the distribution deviation evaluation index.

[0074] For example, a query statement is input into a language model to obtain a query matrix query corresponding to the query statement. The query matrix query is multiplied by the concatenation matrix information and the distribution deviation evaluation index to obtain the text category corresponding to the query statement.

[0075] Please refer to Figure 3 In some embodiments, the splicing matrix information includes a splicing key matrix and a splicing value matrix. The text category corresponding to the query statement is determined based on the query matrix, the splicing matrix information, and the distribution deviation evaluation index. Step S7 includes steps S71 to S73.

[0076] Step S71: Multiply each row vector of the query matrix with each column vector of the spliced ​​key matrix to obtain an attention vector, and then determine a first attention matrix based on the attention vector.

[0077] For example, each row vector of the query matrix is ​​multiplied by each column vector of the spliced ​​key matrix to obtain an attention vector, and then the attention vectors are spliced ​​to obtain the first attention matrix A, as shown in the formula

[0078] As shown in formula 2.

[0079]

[0080] Among them, q i Indicates the value corresponding to the first column of the i-th row in the query matrix, k j It represents the value corresponding to the i-th row and j-th column in the concatenated key matrix, and d represents the hidden state dimension.

[0081] Step S72: Determine a second attention matrix corresponding to the query statement based on the first attention matrix and the distribution deviation evaluation index.

[0082] Exemplarily, the first attention matrix and the cloth deviation evaluation index are multiplied to obtain the second attention matrix corresponding to the query statement.

[0083] In some embodiments, determining the second attention matrix corresponding to the query statement based on the first attention matrix and the distribution deviation evaluation index includes: using the first attention matrix and the distribution deviation evaluation index to determine the target distribution deviation evaluation index corresponding to the attention vector; using the first attention matrix and the target distribution deviation evaluation index to determine the second attention matrix corresponding to the query statement.

[0084] For example, when the query matrix is ​​multiplied by each column vector of the spliced ​​key matrix, each column vector belongs to different example data, and each group of example data corresponds to a corresponding distribution deviation evaluation index. Therefore, the target distribution deviation evaluation index corresponding to the attention vector is determined according to the first attention matrix and the distribution deviation evaluation index, and then the first attention matrix and the target distribution deviation evaluation index are multiplied to obtain the second attention matrix corresponding to the query statement. The specific calculation method is shown in Formula 3.

[0085]

[0086] Among them, q i Indicates the value corresponding to the first column of the i-th row in the query matrix, k j represents the value corresponding to the i-th row and j-th column in the spliced ​​key matrix, d represents the hidden state dimension, z t represents the tth group of sample data, Represents the distribution deviation evaluation index corresponding to the t-th group of sample data.

[0087] For example, there are three groups of example data, and the first key matrix K1, the second key matrix K2, the third key matrix K3, the first value matrix V1, the second value matrix V2, the third value matrix V3, and the first distribution deviation evaluation index P1, the second distribution deviation evaluation index P2, and the third distribution deviation evaluation index P3 are obtained respectively. The query matrix query, the key matrix Kx and the value matrix Vx corresponding to the existing query data, then the spliced ​​key matrix is ​​[K1, K2, K3, Kx], and the spliced ​​value matrix is ​​[V1, V2, V3, Vx]. Then, the target distribution deviation evaluation index after multiplying query with the column vector corresponding to K1 in the spliced ​​key matrix is ​​P1, the target distribution deviation evaluation index after multiplying query with the column vector corresponding to K2 in the spliced ​​key matrix is ​​P2, the target distribution deviation evaluation index after multiplying query with the column vector corresponding to K3 in the spliced ​​key matrix is ​​P3, and the target distribution deviation evaluation index after multiplying query with the column vector corresponding to Kx in the spliced ​​key matrix is ​​1.

[0088] Step S73: Determine the text category corresponding to the query statement based on the second attention matrix and the spliced ​​value matrix.

[0089] Exemplarily, the second attention matrix and the spliced ​​value matrix are matrix multiplied to obtain the corresponding vector, and then the vector is converted into the probability value corresponding to each category according to softmax, and the text category corresponding to the maximum probability value is used as the text category corresponding to the query statement.

[0090] This application overcomes the input length limitation of the context learning task caused by the large language model structure, allowing the task to input a large number of examples and fully absorb examples with different distributions in the downstream sample space. By overcoming the input length limitation of the large language model structure, the context learning task can accommodate a large number of examples and absorb examples with different distributions from the downstream sample space, thereby achieving more sufficient example input. This can better adapt to different data distributions and sample scenarios, and provide more comprehensive and rich input information for the completion of the task. By solving the input length limitation, the context learning task can handle larger-scale and more diverse example data, thereby improving the application ability of the model in actual production environments.

[0091] In addition, the solution of the present application improves the over-sensitivity problem caused by example selection in contextual learning tasks, and improves the stability and robustness of contextual learning tasks. By improving the example selection process in contextual learning tasks, the over-sensitivity problem caused by example selection can be effectively solved, and the stability and robustness of contextual learning tasks can be improved. Not only can it overcome the influence of template selection and example sorting, but it can also consider the relationship between examples, so as to select examples more comprehensively. This improved method can reduce the model's excessive dependence on different examples and improve the consistency and reliability of the model in different scenarios. At the same time, the new method can also introduce indicators such as the distribution deviation coefficient to evaluate the performance of the example group, so as to more finely adjust the weight distribution of the examples to optimize the learning process of the model. This improvement is very critical to improving the performance of contextual learning tasks and the robustness in actual application scenarios, and provides stronger stability and reliability guarantees for the actual production application of the model.

[0092] The distribution deviation coefficient is used to measure the performance of example data, giving more weight to important examples and ensuring that attention is not dispersed. By using the distribution deviation coefficient as a metric for example group performance, the contribution of different example groups can be assessed and more weight can be given to important example groups, ensuring that attention is not overly dispersed. This approach more carefully considers the differences between example groups, allowing contextual learning tasks to more accurately select and utilize information from different example groups, thereby improving model performance. By fully considering the weight distribution of example groups, the model's learning process can be better guided, focusing on key example groups, thereby performing better when handling complex and diverse data distributions. This distribution deviation coefficient-based approach can effectively optimize the performance of contextual learning tasks, improving the model's robustness and generalization. Adding more example data input allows for more "training" of the self-attention module during the forward computation of large language models.

[0093] The text category recognition method provided by this application is applied in the field of financial technology. Taking financial insurance as an example, the content of the conversation between the customer and the salesperson during the insurance recommendation process is obtained, and then the query matrix corresponding to the conversation content is multiplied by the splicing matrix to determine the text category corresponding to the conversation content. In this process, there is no limit on the length of the conversation content. Therefore, it breaks the problem of the existing technology that limits the length of the text and causes the text category recognition accuracy to decrease. Therefore, the text category recognition method provided by this application can effectively identify the real needs of customers in the conversation content, and then recommend insurance or insurance content that is close to their needs to customers, so as to improve customer satisfaction and thus improve the transaction success rate of the company's financial insurance business.

[0094] See also Figure 4 , Figure 4 A text category recognition device 200 is provided in an embodiment of the present application. The text category recognition device 200 includes a data acquisition module 201, a data splicing module 202, a data processing module 203, a data prediction module 204, a data query module 205, a matrix splicing module 206 and a data analysis module 207, wherein the data acquisition module 201 is used to obtain sample data, the sample data including text data and a text category corresponding to the text data, and determine a first splicing strategy and a second splicing strategy for data splicing the text data and the text category; the data splicing module 202 is used to splice the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and continue to splice the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text; the data processing module 203 is used to splice the first spliced ​​text This invention uses a language model to obtain the first matrix information corresponding to the self-attention module of the first spliced ​​text in the language model; a data prediction module 204 is used to perform data prediction on the second spliced ​​text using the language model to obtain a prediction result, and then determine the distribution deviation evaluation index corresponding to the example data according to the prediction result; a data query module 205 is used to obtain a query statement to be predicted, input the query statement into the language model, and obtain the second matrix information corresponding to the query statement in the attention module of the language model; a matrix splicing module 206 is used to splice the first matrix information and the second matrix information to obtain splicing matrix information; a data analysis module 207 is used to obtain a query matrix corresponding to the query statement, and then determine the text category corresponding to the query statement according to the query matrix, the splicing matrix information and the distribution deviation evaluation index.

[0095] In some implementations, during the process of obtaining the sample data, the data acquisition module 201 executes:

[0096] Determine sample data;

[0097] The sample data are randomly grouped to obtain multiple groups of example data.

[0098] In some embodiments, the data splicing module 202 performs the following steps during the process of splicing the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and further splicing the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text:

[0099] Performing data splicing on the text data and the text category according to the first splicing strategy, thereby obtaining a first spliced ​​text corresponding to the sample data;

[0100] A third spliced ​​text is determined according to the second splicing strategy, and the first spliced ​​text and the third spliced ​​text are spliced ​​according to the second splicing strategy to obtain the second spliced ​​text.

[0101] In some embodiments, the first matrix information includes a first key matrix and a first value matrix. In the process of using the language model to obtain the first matrix information corresponding to the self-attention module of the first concatenated text in the language model, the data processing module 203 performs:

[0102] Determining a text length corresponding to the first concatenated text, and when the text length is less than a target text length, padding the first concatenated text to obtain target text data;

[0103] The target text data is input into the language model to obtain the first key matrix and the first value matrix corresponding to the self-attention module of the target text data in the language model.

[0104] In some embodiments, the prediction result includes a prediction category and a prediction probability corresponding to the prediction category. In the process of determining the distribution deviation evaluation index corresponding to the example data based on the prediction result, the data prediction module 204 performs:

[0105] Calculate the relative entropy of the prediction result according to the prediction category and the prediction probability to obtain a corresponding relative entropy result;

[0106] The distribution deviation evaluation index corresponding to the example data is determined according to the relative entropy result.

[0107] In some implementations, the splicing matrix information includes a splicing key matrix and a splicing value matrix. In the process of determining the text category corresponding to the query statement based on the query matrix, the splicing matrix information, and the distribution deviation evaluation index, the data analysis module 207 performs:

[0108] Multiplying each row vector of the query matrix by each column vector of the spliced ​​key matrix to obtain an attention vector, and then determining a first attention matrix according to the attention vector;

[0109] Determine a second attention matrix corresponding to the query statement according to the first attention matrix and the distribution deviation evaluation index;

[0110] Determine the text category corresponding to the query statement according to the second attention matrix and the spliced ​​value matrix.

[0111] In some embodiments, during the process of determining the second attention matrix corresponding to the query statement based on the first attention matrix and the distribution deviation evaluation index, the data analysis module 207 performs:

[0112] Determine a target distribution deviation evaluation index corresponding to the attention vector using the first attention matrix and the distribution deviation evaluation index;

[0113] The first attention matrix and the target distribution deviation evaluation index are used to determine a second attention matrix corresponding to the query statement.

[0114] In some implementations, the text category recognition apparatus 200 is applied to a terminal device.

[0115] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned text category recognition method embodiment, and will not be repeated here.

[0116] See also Figure 5 , Figure 5 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present application.

[0117] like Figure 5 As shown, the terminal device 300 includes a processor 301 and a memory 302, and the processor 301 and the memory 302 are connected via a bus 303, such as an I2C (Inter-integrated Circuit) bus.

[0118] Specifically, the processor 301 is used to provide computing and control capabilities to support the operation of the entire server. The processor 301 can be a central processing unit (CPU), and the processor 301 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0119] Specifically, the memory 302 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0120] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the embodiment of the present application, and does not constitute a limitation on the terminal device to which the embodiment of the present application is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] The processor 301 is configured to run a computer program stored in the memory, and implement the text category recognition method provided in any embodiment of the present application when executing the computer program.

[0122] In some embodiments, the processor 301 is configured to run a computer program stored in a memory, applied to a terminal device, and implement the following steps when executing the computer program:

[0123] Obtaining sample data, the sample data including text data and a text category corresponding to the text data, and determining a first splicing strategy and a second splicing strategy for data splicing of the text data and the text category;

[0124] Splicing the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and further splicing the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text;

[0125] Using the first concatenated text using a language model, obtain first matrix information corresponding to the self-attention module of the first concatenated text in the language model;

[0126] Performing data prediction on the second concatenated text using the language model to obtain a prediction result, and then determining a distribution deviation evaluation index corresponding to the sample data based on the prediction result;

[0127] Obtaining a query statement to be predicted, inputting the query statement into the language model, and obtaining second matrix information corresponding to the query statement in the attention module of the language model;

[0128] Splicing the first matrix information and the second matrix information to obtain a spliced ​​value matrix;

[0129] A query matrix corresponding to the query statement is obtained, and then a text category corresponding to the query statement is determined according to the query matrix, the concatenation matrix information, and the distribution deviation evaluation index.

[0130] In some implementations, during the process of obtaining the sample data, the processor 301 executes:

[0131] Determine sample data;

[0132] The sample data are randomly grouped to obtain multiple groups of example data.

[0133] In some embodiments, during the process of performing data splicing on the example data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the example data, and continuing to splice the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text, the processor 301 executes:

[0134] Performing data splicing on the text data and the text category according to the first splicing strategy, thereby obtaining a first spliced ​​text corresponding to the sample data;

[0135] A third spliced ​​text is determined according to the second splicing strategy, and the first spliced ​​text and the third spliced ​​text are spliced ​​according to the second splicing strategy to obtain the second spliced ​​text.

[0136] In some embodiments, the first matrix information includes a first key matrix and a first value matrix. In the process of using a language model to obtain the first matrix information corresponding to the self-attention module of the first concatenated text in the language model, the processor 301 executes:

[0137] Determining a text length corresponding to the first concatenated text, and when the text length is less than a target text length, padding the first concatenated text to obtain target text data;

[0138] The target text data is input into the language model to obtain the first key matrix and the first value matrix corresponding to the self-attention module of the target text data in the language model.

[0139] In some embodiments, the prediction result includes a prediction category and a prediction probability corresponding to the prediction category. In the process of determining the distribution deviation evaluation index corresponding to the example data based on the prediction result, the processor 301 executes:

[0140] Calculate the relative entropy of the prediction result according to the prediction category and the prediction probability to obtain a corresponding relative entropy result;

[0141] The distribution deviation evaluation index corresponding to the example data is determined according to the relative entropy result.

[0142] In some implementations, the splicing matrix information includes a splicing key matrix and a splicing value matrix. In the process of determining the text category corresponding to the query statement based on the query matrix, the splicing matrix information, and the distribution deviation evaluation index, the processor 301 executes:

[0143] Multiplying each row vector of the query matrix by each column vector of the spliced ​​key matrix to obtain an attention vector, and then determining a first attention matrix according to the attention vector;

[0144] Determine a second attention matrix corresponding to the query statement according to the first attention matrix and the distribution deviation evaluation index;

[0145] Determine the text category corresponding to the query statement according to the second attention matrix and the spliced ​​value matrix.

[0146] In some embodiments, during the process of determining the second attention matrix corresponding to the query statement according to the first attention matrix and the distribution deviation evaluation index, the processor 301 executes:

[0147] Determine a target distribution deviation evaluation index corresponding to the attention vector using the first attention matrix and the distribution deviation evaluation index;

[0148] The first attention matrix and the target distribution deviation evaluation index are used to determine a second attention matrix corresponding to the query statement.

[0149] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the terminal device described above can refer to the corresponding process in the aforementioned text category recognition method embodiment, and will not be repeated here.

[0150] An embodiment of the present application also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any text category recognition method provided in the embodiment of the present application specification.

[0151] The storage medium may be an internal storage unit of the terminal device in the aforementioned embodiment, such as a terminal device memory. The storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., provided on the terminal device.

[0152] It will be appreciated by those skilled in the art that all or some of the steps in the method disclosed above, and the functional modules / units in the device can be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media generally embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0153] It should be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0154] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.< / pad>

Claims

1. A method for identifying text categories, characterized in that: The method comprises: Obtaining sample data, the sample data including text data and a text category corresponding to the text data, and determining a first splicing strategy and a second splicing strategy for data splicing of the text data and the text category; Splicing the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and further splicing the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text; Using the first concatenated text using a language model, obtain first matrix information corresponding to the self-attention module of the first concatenated text in the language model; Performing data prediction on the second concatenated text using the language model to obtain a prediction result, and then determining a distribution deviation evaluation index corresponding to the sample data based on the prediction result; Obtaining a query statement to be predicted, inputting the query statement into the language model, and obtaining second matrix information corresponding to the query statement in the attention module of the language model; Splicing the first matrix information and the second matrix information to obtain spliced ​​matrix information; A query matrix corresponding to the query statement is obtained, and then a text category corresponding to the query statement is determined according to the query matrix, the concatenation matrix information, and the distribution deviation evaluation index.

2. The method according to claim 1, characterized in that The obtaining of sample data includes: Determine sample data; The sample data are randomly grouped to obtain multiple groups of example data.

3. The method according to claim 1, characterized in that The step of splicing the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and further splicing the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text includes: Performing data splicing on the text data and the text category according to the first splicing strategy, thereby obtaining a first spliced ​​text corresponding to the sample data; A third spliced ​​text is determined according to the second splicing strategy, and the first spliced ​​text and the third spliced ​​text are spliced ​​according to the second splicing strategy to obtain the second spliced ​​text.

4. The method according to claim 1, wherein The first matrix information includes a first key matrix and a first value matrix. The first matrix information corresponding to the self-attention module of the first concatenated text in the language model is obtained by using the language model, including: Determining a text length corresponding to the first concatenated text, and when the text length is less than a target text length, padding the first concatenated text to obtain target text data; The target text data is input into the language model to obtain the first key matrix and the first value matrix corresponding to the self-attention module of the target text data in the language model.

5. The method according to claim 1, wherein The prediction result includes a prediction category and a prediction probability corresponding to the prediction category. Determining a distribution deviation evaluation index corresponding to the example data based on the prediction result includes: Calculate the relative entropy of the prediction result according to the prediction category and the prediction probability to obtain a corresponding relative entropy result; The distribution deviation evaluation index corresponding to the example data is determined according to the relative entropy result.

6. The method according to claim 1, characterized in that The splicing matrix information includes a splicing key matrix and a splicing value matrix. Determining the text category corresponding to the query statement according to the query matrix, the splicing matrix information, and the distribution deviation evaluation index includes: Multiplying each row vector of the query matrix by each column vector of the spliced ​​key matrix to obtain an attention vector, and then determining a first attention matrix according to the attention vector; Determine a second attention matrix corresponding to the query statement according to the first attention matrix and the distribution deviation evaluation index; Determine the text category corresponding to the query statement according to the second attention matrix and the spliced ​​value matrix.

7. The method according to claim 6, characterized in that The determining, according to the first attention matrix and the distribution deviation evaluation index, a second attention matrix corresponding to the query statement includes: Determine a target distribution deviation evaluation index corresponding to the attention vector using the first attention matrix and the distribution deviation evaluation index; The first attention matrix and the target distribution deviation evaluation index are used to determine a second attention matrix corresponding to the query statement.

8. A text category recognition device, characterized in that: include: A data acquisition module is used to obtain sample data, the sample data including text data and a text category corresponding to the text data, and determine a first splicing strategy and a second splicing strategy for data splicing of the text data and the text category; a data splicing module, configured to splice the sample data according to the first splicing strategy to obtain a first spliced ​​text corresponding to the sample data, and further splice the first spliced ​​text according to the second splicing strategy to obtain a second spliced ​​text; A data processing module, configured to use a language model to obtain first matrix information corresponding to a self-attention module of the first concatenated text in the language model; a data prediction module, configured to perform data prediction on the second concatenated text using the language model to obtain a prediction result, and then determine a distribution deviation evaluation index corresponding to the sample data based on the prediction result; A data query module, configured to obtain a query statement to be predicted, input the query statement into the language model, and obtain second matrix information corresponding to the query statement in the attention module of the language model; a matrix splicing module, configured to splice the first matrix information and the second matrix information to obtain spliced ​​matrix information; The data analysis module is used to obtain a query matrix corresponding to the query statement, and then determine the text category corresponding to the query statement based on the query matrix, the splicing matrix information and the distribution deviation evaluation index.

9. A terminal device, characterized in that: The terminal device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the text category recognition method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that When the computer-readable storage medium is executed by one or more processors, the one or more processors are enabled to perform the steps of the text category recognition method according to any one of claims 1 to 7.

Images