Text processing method, device and computer readable storage medium

Abstract

The present invention discloses a text processing method, device and computer-readable storage medium, and relates to the field of natural language processing technology. The text processing method includes: converting the text associated with an order into a text vector; inputting the text vector into a pre-trained text classification model, the text classification model includes an encoder, a decoder and a mapping module, the number of network layer modules in each of the encoder and the decoder is less than a preset value, the network layer module includes a first network layer module and a second network layer module, and the first network layer module has a residual structure and the second network layer module is a serial structure, and the mapping module is used to map the output of the decoder to a classification result; obtain the classification result output by the text classification model, wherein the classification result indicates whether an early warning is generated for an order. The embodiment of the present invention improves the computational efficiency of the model without losing the performance of the model, and is suitable for order early warning scenarios with high real-time requirements and large computational complexity.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text processing method, device and computer-readable storage medium. Background Art

[0002] With the advent of the Internet age, online shopping has gradually become one of the main ways for people to shop. However, online shopping will also cause some problems, such as the discrepancy between the picture and the actual product, the damage of the product due to logistics reasons, the sudden change of the user's mind, etc. If the potential problems of the user's order cannot be discovered in time, it will lead to a poor shopping experience for the user, and even cause the user to vent his dissatisfaction on other platforms. As a result, not only the rights of consumers are damaged, but also the shopping platform cannot discover the problems in the system in time, and even have a serious impact on the company's reputation and image. If the problems that trouble users can be effectively solved in a timely and effective manner before the user's public opinion is exposed, it will avoid losses to users.

[0003] In related technologies, warnings are usually issued by manually reviewing texts. Alternatively, experts design relevant rules to manually find features that can determine the development trend of public opinion, and then extract corresponding features from the text to cluster the text or perform classification processing based on support vector machines (SVM) to decide whether to issue a warning based on the classification results. Summary of the invention

[0004] After analysis, the inventors found that the text processing methods in the relevant technologies all introduced artificial and uncontrollable factors, making it difficult to effectively provide order warnings.

[0005] A technical problem to be solved by the embodiments of the present invention is: how to provide an effective order early warning solution so as to timely discover problems existing in the system.

[0006] According to a first aspect of some embodiments of the present invention, there is provided a text processing method, comprising: converting text associated with an order into a text vector; inputting the text vector into a pre-trained text classification model, wherein the text classification model comprises an encoder, a decoder and a mapping module, the number of network layer modules in each of the encoder and the decoder is less than a preset value, the network layer modules comprise a first network layer module and a second network layer module, and the first network layer module has a residual structure and the second network layer module is a serial structure, and the mapping module is used to map the output of the decoder into a classification result; obtaining a classification result output by the text classification model, wherein the classification result indicates whether a warning is generated for the order.

[0007] In some embodiments, the preset value is 6.

[0008] In some embodiments, in the encoder and the decoder, from the input end to the output end, the first network layer module, the second network layer module, the first network layer module, and the second network layer module are included in sequence.

[0009] In some embodiments, each network layer module includes a self-attention mechanism layer, a first vector addition and normalization layer, a feedforward neural network layer, and a second vector addition and normalization layer.

[0010] In some embodiments: in the first network layer module, the input of the first vector addition and normalization layer includes the input of the first network layer module and the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer includes the output of the first vector addition and normalization layer and the output of the feedforward neural network layer; and, in the second network layer module, the input of the first vector addition and normalization layer is the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer is the output of the feedforward neural network layer.

[0011] In some embodiments, converting the text associated with an order into a text vector includes: determining an embedding vector for each word in the text associated with the order; and constructing a text vector using the embedding vector of each word, wherein the number of dimensions of the text vector is equal to a preset text length multiplied by the number of dimensions of the embedding vector corresponding to each word in the text.

[0012] In some embodiments, the order-associated text includes customer service text data, user text data, and system text data associated with the order.

[0013] In some embodiments, the text processing method further includes: when the classification result indicates that a warning is generated for the order, obtaining user data corresponding to the user of the order; and determining the warning level of the order based on the user data.

[0014] In some embodiments, determining the warning level of an order based on user data includes: determining the warning level corresponding to historical data of each category in the user data; and determining the warning level with the highest frequency as the warning level of the order.

[0015] In some embodiments, the text processing method also includes: when the warning level of the order is low risk, pushing a questionnaire or smart phone to the user, the questionnaire or smart phone including solutions corresponding to the order for the user to choose; or, when the warning level of the order is medium risk, communicating with the user through an intelligent voice robot, and determining the solution corresponding to the order based on the user's response; or, when the warning level of the order is high risk, transferring the order to manual customer service.

[0016] In some embodiments, the text processing method also includes: obtaining historical data, wherein the historical data includes text associated with an order before an alert is generated; converting the text in the historical data into text vectors for training; training a text classification model using the text vectors for training; and adjusting the parameters of the text classification model based on the classification results of the text classification model and the alert results corresponding to the text vectors for training.

[0017] According to a second aspect of some embodiments of the present invention, there is provided a text processing device, comprising: a conversion module, configured to convert text associated with an order into a text vector; an input module, configured to input the text vector into a pre-trained text classification model, wherein the text classification model comprises an encoder, a decoder and a mapping module, the number of network layer modules in each of the encoder and the decoder is less than a preset value, the network layer modules comprise a first network layer module and a second network layer module, and the first network layer module has a residual structure and the second network layer module is a serial structure, the mapping module is used to map the output of the decoder into a classification result; an early warning module, configured to obtain a classification result output by the text classification model, wherein the classification result indicates whether an early warning is generated for the order.

[0018] In some embodiments, the text processing device also includes: an early warning level determination module, which is configured to obtain user data corresponding to the user of the order when the classification result indicates that an early warning is generated for the order; and determine the early warning level of the order based on the user data.

[0019] In some embodiments, the text processing device also includes: an early warning processing module, which is configured to push a questionnaire or a smart phone to the user when the early warning level of the order is low risk, and the questionnaire or the smart phone includes solutions corresponding to the order for the user to choose; or, when the early warning level of the order is medium risk, communicate with the user through an intelligent voice robot and determine the solution corresponding to the order based on the user's response; or, when the early warning level of the order is high risk, transfer the order to manual customer service.

[0020] In some embodiments, the text processing device also includes: a training module configured to obtain historical data, wherein the historical data includes text associated with an order before an alert is generated; converting the text in the historical data into a text vector for training; training a text classification model using the text vector for training; and adjusting the parameters of the text classification model based on the classification results of the text classification model and the alert results corresponding to the text vector for training.

[0021] According to a third aspect of some embodiments of the present invention, there is provided a text processing device, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute any one of the aforementioned text processing methods based on instructions stored in the memory.

[0022] According to a fourth aspect of some embodiments of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements any one of the aforementioned text processing methods when executed by a processor.

[0023] Some embodiments of the above invention have the following advantages or beneficial effects. In the embodiments of the present invention, the encoder and the decoder have fewer network layer modules, so the network depth is shallow. In this case, no residual structure is set for each network layer module, so that the computational efficiency of the model can be improved without losing the model performance, which is suitable for order warning scenarios with high real-time requirements and large computational workload. Thus, the accuracy and efficiency of order warning are improved.

[0024] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 A schematic flow chart of a text processing method according to some embodiments of the present invention is shown.

[0027] Figure 2A and Figure 2B The structural schematic diagrams of the first network layer module and the second network layer module are exemplarily shown respectively.

[0028] Figure 3 A schematic flow chart of a method for training a text classification model according to some embodiments of the present invention is shown.

[0029] Figure 4 A schematic flow chart of an order early warning method according to some embodiments of the present invention is shown.

[0030] Figure 5 A schematic flow chart of a warning processing method according to some embodiments of the present invention is shown.

[0031] Figure 6A schematic structural diagram of a text processing device according to some embodiments of the present invention is shown.

[0032] Figure 7 A schematic structural diagram of a text processing device according to some other embodiments of the present invention is shown.

[0033] Figure 8 A schematic structural diagram of a text processing device according to some further embodiments of the present invention is shown. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] The relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0036] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0037] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0038] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0039] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0040] The BERT (Bidirectional Encoder Representation from Transformers, deep bidirectional pre-training based on semantic understanding) algorithm has good results in the field of natural language processing, and the Transformer network in the BERT algorithm is a key component of the algorithm. In the related art, the Transformer network utilizes the attention mechanism and is mainly composed of an encoder and a decoder, wherein the encoder and the decoder each include 6 network layer modules. Due to the large network depth, in order to avoid the problem of gradient disappearance, each network layer module in the encoder and the decoder has a residual structure. In addition, the BERT algorithm in the related art is usually used to process large sections of text, and the correlation between contexts in such texts is relatively strong. Therefore, when processing the text into a vector, in addition to the embedding vector that expresses the meaning of the text itself, a position vector is also added. Thus, in the Transformer network in the related art, the vector input to the encoder is composed of an embedding vector and a position vector.

[0041] When warning an order, it is necessary to promptly discover the potential risks of the order. Therefore, the real-time requirements for data processing are relatively high. In order to improve the processing performance and efficiency of the system, the present invention improves the Transformer network to adapt to the application scenario of order warning. Figure 1 An embodiment of the text processing method of the present invention is described.

[0042] Figure 1 FIG. 2 shows a flow chart of a text processing method according to some embodiments of the present invention. Figure 1 As shown, the text processing method of this embodiment includes steps S102 to S106.

[0043] In step S102, the text associated with the order is converted into a text vector. In some embodiments, the text associated with the order includes one or more of customer service text data, user text data, and system text data associated with the order.

[0044] After-sales messages, notes and other customer service text data are filled in by professional customer service personnel. Their format and content are more standardized, and the qualitative and description of the problem are more accurate. This type of data makes the model's prediction results more accurate. Furthermore, in order to make the data as free as possible from the influence of unilateral factors and human subjective factors, user text data (such as user messages, comments, and conversation texts) and system text data (such as system logs) can also be added during the prediction, so that the data type is richer and the prediction results are more accurate.

[0045] In some embodiments, when the amount of text data associated with an order is large, the latest text with a preset amount or the latest text with a preset amount of data is obtained, so that the selected data is closer to the current state.

[0046] In some embodiments, in the process of converting text into a text vector, the text is first segmented, and then each word is converted into a vector form that can be recognized by a computer, thereby forming a text vector. In order to facilitate unified processing by the text classification model, the text is adjusted to a preset length before vector conversion. If the original text exceeds the preset length, the text is truncated; if the length of the original text does not reach the preset length, the text is padded with 0.

[0047] In some embodiments, an embedding vector of each word in the text associated with the order is determined; and a text vector is constructed using the embedding vector of each word, wherein the number of dimensions of the text vector is equal to the number of dimensions of the embedding vector corresponding to each word in the text multiplied by the preset text length. That is, the text vector of this embodiment does not include a position vector.

[0048] After analysis, the inventors found that for texts with strong context relevance, in order to mine the contextual relationship of words, the Transformer network used position vectors during encoding. However, in the scenario of order warnings, since the text content is relatively scattered, such as mostly customer service and user messages, and when the text includes multiple types of data such as customer service text data, user text data, and system text data, there is no strict contextual association relationship between these data. Therefore, in order to further improve computing efficiency, only embedded vectors can be used during encoding instead of position vectors. In this way, the computing efficiency of order warnings is further improved, and high real-time applications can be achieved with fewer computing resources.

[0049] In step S104, the text vector is input into a pre-trained text classification model.

[0050] In step S106, a classification result output by the text classification model is obtained, wherein the classification result indicates whether an early warning is generated for the order.

[0051] In some embodiments, the early warning task is executed regularly, for example, the text associated with the order is periodically obtained and the corresponding classification results are obtained, so as to timely discover potential problems in the system without significantly affecting the system performance.

[0052] The text classification model includes an encoder, a decoder and a mapping module. The number of network layer modules in each of the encoder and the decoder is less than a preset value. In some embodiments, the preset value is 6, that is, the number of network layer modules used in the encoder and the decoder of this embodiment is less than the number of network layer modules used in the encoder and the decoder in the Transformer network.

[0053] The network layer module includes a first network layer module and a second network layer module, wherein the first network layer module has a residual structure, and the second network layer module is a serial structure, that is, does not have a residual structure.

[0054] The mapping module is used to map the output of the decoder to a classification result. In some embodiments, the mapping module includes a linear layer and a softmax layer. For example, the values ​​of each dimension of the output vector of the decoder are calculated using the parameters preset in the mapping module, and the classification result is determined based on the comparison result between the calculation result and the preset threshold.

[0055] Since the encoder and decoder have fewer network layer modules in this embodiment, the network depth is shallower than that of the Transformer network. In this case, this embodiment does not set a residual structure for each network layer module, so that the computational efficiency of the model can be improved without losing the model performance, which is suitable for order warning scenarios with high real-time requirements and large computational workload. Thus, the accuracy and efficiency of order warnings are improved.

[0056] The structure of the text classification model used in some embodiments is further described below.

[0057] In some embodiments, each network layer module includes a self-attention mechanism layer, a first vector addition and normalization layer, a feedforward neural network layer, and a second vector addition and normalization layer.

[0058] The self-attention mechanism layer is used to multiply the input vector of this layer by three preset weight matrices to obtain three vectors, which are represented by q, k, and v respectively. Then, we calculate The output of the self-attention mechanism layer is determined based on the calculation results, where softmax(·) represents the activation function applied to the content in the brackets, and d k Represents the dimension of vector k.

[0059] Figure 2A and Figure 2B The structural schematic diagrams of the first network layer module and the second network layer module are exemplarily shown respectively.

[0060] In some embodiments, in the first network layer module, the input of the first vector addition and normalization layer includes the input of the first network layer module and the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer includes the output of the first vector addition and normalization layer and the output of the feedforward neural network layer. That is, a residual structure is used when performing vector addition and normalization.

[0061] In the second network layer module, the input of the first vector addition and normalization layer is the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer is the output of the feedforward neural network layer. Thus, the second network layer module uses a serial structure.

[0062] In some embodiments, in order to prevent the input data from falling into the saturation region of the activation function, the input of each network layer module is converted into data with a mean of 0 and a variance of 1.

[0063] In addition to the network layer modules, the text classification model can also include a maximum pooling layer, a fully connected layer, a dropout layer, and so on.

[0064] The maximum pooling layer is used to take the point with the largest value in the local receptive field of the feature to replace the local receptive field, thereby reducing the model size, increasing the calculation speed, and improving the robustness of the extracted features.

[0065] In the fully connected layer, each neuron node is connected to all the network nodes in the previous layer to summarize the features extracted previously and map the distributed feature representation learned in the previous layer to the sample label space.

[0066] The Dropout layer is used to temporarily discard neural network units from the network with a certain probability during the training process of the neural network model, thereby effectively preventing the model from overfitting.

[0067] Reference below Figure 3 An embodiment of the text classification model training method of the present invention is described.

[0068] Figure 3 FIG. 2 is a flow chart showing a method for training a text classification model according to some embodiments of the present invention. Figure 3 As shown, the training method of this embodiment includes steps S302 to S308.

[0069] In step S302, historical data is obtained, wherein the historical data includes text associated with the order before the warning is generated.

[0070] For example, for orders that have generated warnings, if the warning is generated at time X, then the associated text in the training data only takes the text before time X; while for orders that have not generated warnings, the text at any time can be taken.

[0071] In step S304, the text in the historical data is converted into a text vector for training.

[0072] In step S306, the text classification model is trained using the text vectors used for training.

[0073] In step S308, the parameters of the text classification model are adjusted according to the classification result of the text classification model and the warning result corresponding to the text vector used for training.

[0074] The above embodiment uses the text associated with the order before the warning is generated when selecting training data, thereby avoiding the interference of the text after the warning is generated on the warning result, thereby improving the accuracy of model training.

[0075] After determining that an alert is required through the text classification model, the alert level can also be determined. Figure 4 An embodiment of the order early warning method of the present invention is described.

[0076] Figure 4 FIG. 2 shows a flow chart of an order warning method according to some embodiments of the present invention. Figure 4 As shown, the order warning method of this embodiment includes steps S402 to S404.

[0077] In step S402, when the classification result indicates that a warning is generated for the order, user data corresponding to the user of the order is obtained.

[0078] In step S404, the warning level of the order is determined according to the user data.

[0079] Therefore, the text classification model is used to determine whether to issue an early warning, and after the early warning is generated, the specific early warning level is determined based on historical data. This can adapt to user habits and provide early warning results more accurately.

[0080] In some embodiments, the warning level corresponding to each category of historical data in the user data is determined; the warning level with the highest frequency is determined as the warning level of the order. Table 1 exemplarily shows the determination methods corresponding to multiple categories of historical data.

[0081] Table 1

[0082]

[0083] For example, a user has a history of reporting, malicious ordering, and dispute records, and the user portrait score is lower than the low threshold, that is, the above four categories all correspond to high risk levels; its historical after-sales satisfaction is between the high and low thresholds, and the user level and stickiness are medium, that is, these two categories correspond to medium risk levels. Based on the proportion of high, medium, and low risks, the order that generates the warning corresponding to the user is determined to be a high risk level.

[0084] Therefore, the warning level of an order can be measured in multiple dimensions, making the judgment result more accurate.

[0085] Reference below Figure 5 An embodiment of the method for processing after an order warning is generated according to the present invention is described.

[0086] Figure 5 FIG. 2 is a flow chart showing a method for processing an early warning according to some embodiments of the present invention. Figure 5 As shown, the warning processing method of this embodiment includes steps S502 to S508.

[0087] In step S502, in response to generating an alert for an order, an alert level of the order is determined.

[0088] In step S504, when the warning level of the order is low risk, a questionnaire or a smart phone is pushed to the user, and the questionnaire or the smart phone includes solutions corresponding to the order that can be selected by the user.

[0089] For low-risk orders, we mainly use intelligent means to handle them, mainly through questionnaires or smart phones to understand users' questions about the purchase and after-sales service. At the same time, we also set up corresponding solutions, such as preferential policies, compensation policies, return and exchange channels and other processing channels.

[0090] In response to the solution selected by the user through the terminal or telephone button feedback, the backend server triggers the intelligent review process. Through the intelligent processing process, the risk of public opinion exposure is avoided, while also saving manpower and time costs, helping users solve problems more efficiently. If the user does not respond to the questionnaire or phone call, the manual customer service will be triggered to follow up.

[0091] In step S506, when the warning level of the order is medium risk, the intelligent voice robot communicates with the user and determines the solution corresponding to the order based on the user's response.

[0092] For orders with medium risk levels, the corresponding users are communicated with through intelligent voice robots. The intelligent voice robots are pre-loaded with several questions, and play the next question to the user one by one according to the user's response, and record the user's answer to each question. This is a more efficient intelligent questionnaire solution, which saves users time in reading and selecting questions, and allows users to complete relevant surveys directly through dialogue.

[0093] When the user selects a solution, the backend server triggers the manual review process, and after the review is completed, the compensation process is triggered. If the user does not respond to the intelligent voice robot, the task will be forwarded to the manual customer service. This method can more effectively prevent medium-risk users from developing into high-risk users.

[0094] In step S508, when the warning level of the order is high risk, the order is transferred to manual customer service.

[0095] Through the above embodiments, when an order generates an early warning, corresponding means can be used to handle it according to the early warning level of the order, thereby improving the efficiency of problem handling.

[0096] Reference below Figure 6 An embodiment of a text processing apparatus according to the present invention is described.

[0097] Figure 6 FIG. 4 is a schematic diagram showing the structure of a text processing device according to some embodiments of the present invention. Figure 6 As shown, the text processing device 60 of this embodiment includes: a conversion module 610, configured to convert the text associated with the order into a text vector; an input module 620, configured to input the text vector into a pre-trained text classification model, wherein the text classification model includes an encoder, a decoder and a mapping module, the number of network layer modules in each of the encoder and the decoder is less than a preset value, the network layer module includes a first network layer module and a second network layer module, and the first network layer module has a residual structure and the second network layer module is a serial structure, and the mapping module is used to map the output of the decoder to a classification result; an early warning module 630, configured to obtain a classification result output by the text classification model, wherein the classification result indicates whether an early warning is generated for the order.

[0098] In some embodiments, the preset value is 6.

[0099] In some embodiments, in the encoder and the decoder, from the input end to the output end, the first network layer module, the second network layer module, the first network layer module, and the second network layer module are included in sequence.

[0100] In some embodiments, each network layer module includes a self-attention mechanism layer, a first vector addition and normalization layer, a feedforward neural network layer, and a second vector addition and normalization layer.

[0101] In some embodiments: in the first network layer module, the input of the first vector addition and normalization layer includes the input of the first network layer module and the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer includes the output of the first vector addition and normalization layer and the output of the feedforward neural network layer; and, in the second network layer module, the input of the first vector addition and normalization layer is the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer is the output of the feedforward neural network layer.

[0102] In some embodiments, the conversion module 610 is further configured to determine an embedding vector for each word in the text associated with the order; and construct a text vector using the embedding vector of each word, wherein the number of dimensions of the text vector is equal to a preset text length multiplied by the number of dimensions of the embedding vector corresponding to each word in the text.

[0103] In some embodiments, the order-associated text includes customer service text data, user text data, and system text data associated with the order.

[0104] In some embodiments, the text processing device 60 also includes: an alert level determination module 640, which is configured to obtain user data corresponding to the user of the order when the classification result indicates that an alert is generated for the order; and determine the alert level of the order based on the user data.

[0105] In some embodiments, the warning level determination module 640 is further configured to determine the warning level corresponding to each category of user data in the user data; and determine the warning level with the highest frequency as the warning level of the order.

[0106] In some embodiments, the text processing device 60 also includes: an early warning processing module 650, which is configured to push a questionnaire or a smart phone to the user when the early warning level of the order is low risk, and the questionnaire or the smart phone includes solutions corresponding to the order that can be selected by the user; or, when the early warning level of the order is medium risk, communicate with the user through an intelligent voice robot and determine the solution corresponding to the order based on the user's response; or, when the early warning level of the order is high risk, transfer the order to manual customer service.

[0107] In some embodiments, the text processing device 60 also includes: a training module 660, configured to obtain historical data, wherein the historical data includes text associated with an order before an alert is generated; converting the text in the historical data into a text vector for training; training a text classification model using the text vector for training; and adjusting the parameters of the text classification model according to the classification results of the text classification model and the alert results corresponding to the text vector for training.

[0108] Figure 7 FIG. 2 shows a schematic diagram of the structure of a text processing device according to some other embodiments of the present invention. Figure 7 As shown, the text processing device 70 of this embodiment includes: a memory 710 and a processor 720 coupled to the memory 710 , and the processor 720 is configured to execute the text processing method in any one of the aforementioned embodiments based on instructions stored in the memory 710 .

[0109] The memory 710 may include, for example, a system memory, a fixed non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs.

[0110] Figure 8 FIG. 4 shows a schematic diagram of the structure of a text processing device according to some other embodiments of the present invention. Figure 8 As shown, the text processing device 80 of this embodiment includes: a memory 810 and a processor 820, and may also include an input / output interface 830, a network interface 840, a storage interface 850, etc. These interfaces 830, 840, 850 and the memory 810 and the processor 820 may be connected, for example, via a bus 860. Among them, the input / output interface 830 provides a connection interface for input / output devices such as a display, a mouse, a keyboard, and a touch screen. The network interface 840 provides a connection interface for various networked devices. The storage interface 850 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0111] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the program implements any one of the aforementioned text processing methods when executed by a processor.

[0112] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable non-transient storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A text processing method, include: Converting the text associated with the order into a text vector, wherein the text vector includes an embedding vector and does not include a position vector; The text vector is input into a pre-trained text classification model, wherein the text classification model includes an encoder, a decoder and a mapping module, the number of network layer modules in each of the encoder and the decoder is less than a preset value, the network layer module includes a first network layer module and a second network layer module, in the encoder and the decoder, from the input end to the output end, the first network layer module, the second network layer module, the first network layer module, the second network layer module, and the first network layer module have a residual structure, the second network layer module is a serial structure and does not have a residual structure, each network layer module includes a self-attention mechanism layer, a first vector addition and A normalization layer, a feedforward neural network layer, and a second vector addition and normalization layer. In the first network layer module, the input of the first vector addition and normalization layer includes the input of the first network layer module and the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer includes the output of the first vector addition and normalization layer and the output of the feedforward neural network layer; in the second network layer module, the input of the first vector addition and normalization layer is the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer is the output of the feedforward neural network layer. The mapping module is used to map the output of the decoder to a classification result; A classification result output by the text classification model is obtained, wherein the classification result indicates whether a warning is generated for the order.

2. The text processing method according to claim 1, in, The preset value is 6.

3. The text processing method according to claim 1, in, The step of converting the text associated with the order into a text vector comprises: Determining an embedding vector for each word in the text associated with the order; The text vector is constructed using the embedding vector of each word, wherein the dimension of the text vector is equal to the preset text length multiplied by the dimension of the embedding vector corresponding to each word in the text.

4. The text processing method according to claim 1, in, The text associated with the order includes customer service text data, user text data and system text data associated with the order.

5. The text processing method according to claim 1, further comprising: include: When the classification result indicates that a warning is generated for the order, obtaining user data corresponding to a user of the order; The warning level of the order is determined according to the user data.

6. The text processing method according to claim 5, in, Determining the warning level of the order according to the user data includes: Determining a warning level corresponding to each category of user data in the user data; The warning level with the highest frequency of occurrence is determined as the warning level of the order.

7. The text processing method according to claim 5, further comprising: include: In the case where the warning level of the order is low risk, a questionnaire or a smart phone is pushed to the user, wherein the questionnaire or the smart phone includes a solution corresponding to the order that can be selected by the user; or When the warning level of the order is medium risk, the intelligent voice robot communicates with the user and determines the solution corresponding to the order according to the user's response; or, When the warning level of the order is high risk, the order is transferred to manual customer service.

8. The text processing method according to claim 1, further comprising: include: Acquire historical data, wherein the historical data includes text associated with the order before the warning is generated; Converting the text in the historical data into a text vector for training; Using the text vector for training to train the text classification model; According to the classification result of the text classification model and the warning result corresponding to the text vector used for training, the parameters of the text classification model are adjusted.

9. A text processing device, include: A conversion module, configured to convert text associated with an order into a text vector, wherein the text vector includes an embedding vector but does not include a position vector; An input module is configured to input the text vector into a pre-trained text classification model, wherein the text classification model includes an encoder, a decoder and a mapping module, the number of network layer modules in each of the encoder and the decoder is less than a preset value, the network layer module includes a first network layer module and a second network layer module, in the encoder and the decoder, from the input end to the output end, the first network layer module, the second network layer module, the first network layer module, the second network layer module, and the first network layer module have a residual structure, the second network layer module is a serial structure and does not have a residual structure, each network layer module includes a self-attention mechanism layer, a first vector module, and a second network layer module. A vector addition and normalization layer, a feedforward neural network layer, and a second vector addition and normalization layer. In the first network layer module, the input of the first vector addition and normalization layer includes the input of the first network layer module and the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer includes the output of the first vector addition and normalization layer and the output of the feedforward neural network layer; in the second network layer module, the input of the first vector addition and normalization layer is the output of the self-attention mechanism layer, and the input of the second vector addition and normalization layer is the output of the feedforward neural network layer. The mapping module is used to map the output of the decoder to a classification result; The warning module is configured to obtain a classification result output by the text classification model, wherein the classification result indicates whether a warning is generated for the order.

10. The text processing device according to claim 9, further comprising: include: A warning level determination module, configured to obtain user data corresponding to a user of the order when the classification result indicates that a warning is generated for the order; The warning level of the order is determined according to the user data.

11. The text processing device according to claim 10, further comprising: include: The early warning processing module is configured to push a questionnaire or a smart phone to the user when the early warning level of the order is low risk, and the questionnaire or the smart phone includes a solution corresponding to the order that the user can choose; or, when the early warning level of the order is medium risk, communicate with the user through an intelligent voice robot and determine the solution corresponding to the order based on the user's response; or, when the early warning level of the order is high risk, transfer the order to manual customer service.

12. The text processing device according to claim 9, further comprising: include: The training module is configured to obtain historical data, wherein the historical data includes text associated with an order before an early warning is generated; convert the text in the historical data into a text vector for training; use the text vector for training to train the text classification model; and adjust the parameters of the text classification model according to the classification result of the text classification model and the early warning result corresponding to the text vector for training.

13. A text processing device, include: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the text processing method according to any one of claims 1 to 8 based on instructions stored in the memory.

14. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the text processing method according to any one of claims 1 to 8 is implemented.

Images