Text robot training method, system, device and medium based on deep learning

Abstract

The application discloses a kind of based on deep learning text robot training method, system, equipment and medium, the method includes: based on original intention arrangement pre-set quantity of seed corpus, seed corpus is introduced into text robot knowledge base;Identify the problem data in all intents, and correct problem data;All intents include original intention and seed corpus;Batch expansion is written to all intents;Identify whether the confused corpus is contained in the intent after expansion;When containing, correct confused corpus;Text robot model is trained by original intention, seed corpus and the corrected intent after expansion;Verify whether text robot model meets expectation;When not meeting expectation, by expanding to single intent or adjusting or expanding to single corpus, text robot model is trained again, until text robot model meets expectation.Processing scheme of the present application can efficiently and accurately train text robot.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method, system, device and medium for training a text robot based on deep learning. Background Technology

[0002] The operation of a text robot includes setup and optimization during the cold start phase, improvement of uncovered scenarios after launch, corpus supplementation, and optimization of question-and-answer performance, in order to ensure that the robot has a good generalization effect in specific business scenarios and achieves the expected question-and-answer accuracy.

[0003] Contrary to popular belief, the effectiveness of robots is not inherent when they leave the factory. Instead, it requires significant human investment in construction and continuous optimization to achieve the desired results. In traditional robot operation methods, the setup, optimization, and scenario refinement of robots all rely on the experience of robot trainers, who manually complete tasks such as supplementing corpora with individual intentions, testing and verifying individual questions, and conducting quality checks on each conversation.

[0004] In recent years, many companies have found that their AI (Artificial Intelligence) robots, after purchasing them, have failed to meet initial expectations. The fundamental reason lies in the fact that operating these robots requires a significant investment of human resources. These resources must understand the company's business operations and possess a deep understanding of the robot's operating principles. Only by meeting these three conditions can a robot be effectively operated and its question-and-answer capabilities meet real business needs. Meeting these three conditions is extremely difficult for companies, which is one of the main reasons why robot products are not truly adopted and used effectively in enterprises.

[0005] Therefore, it is evident that the existing text-based robot training methods described above still have inconveniences and shortcomings in use, and urgently need further improvement. Creating a new text-based robot training method has become a pressing goal for the industry. Summary of the Invention

[0006] In view of this, the present disclosure provides a deep learning-based text robot training method, which at least partially solves the problems existing in the prior art.

[0007] In a first aspect, embodiments of this disclosure provide a deep learning-based text robot training method, the method comprising the following steps:

[0008] Based on the original intent, a preset number of seed corpora are organized, and the seed corpora are imported into the text robot knowledge base;

[0009] Identify problematic data in all intents and correct the problematic data; wherein, all intents include the original intent and the seed corpus;

[0010] Batch expand all the aforementioned intents to obtain the expanded intents;

[0011] Identify whether the expanded intent contains obfuscated material; if it does, modify the obfuscated material to obtain the expanded modified intent;

[0012] The text robot model is trained using the original intent, the seed corpus, and the expanded and corrected intent.

[0013] Verify whether the text robot model meets expectations; wherein, when the text robot model does not meet expectations, retrain the text robot model by expanding a single intent or adjusting or expanding a single corpus until the text robot model meets expectations.

[0014] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0015] The text robot model that meets the expected specifications will be released online.

[0016] The text robot model cannot recognize user questions, and hierarchical clustering is used to perform cluster analysis on the user questions to obtain the cluster analysis results.

[0017] Suggestions are provided based on the clustering analysis results; the suggestions include creating new intents based on the clustering analysis results or adding the clustering analysis results to the corpus of existing intents;

[0018] Whenever an intent is created or modified, problematic data in all intents is identified and corrected.

[0019] According to one specific implementation of this disclosure, the question data in all intents includes at least one of: FAQ answer similarity data, confusion intent data, and confusion corpus data.

[0020] According to a specific implementation of an embodiment of this disclosure, the step of batch expanding all the intentions includes:

[0021] The original data is cleaned, clustered, and resampled to construct training data; the original data is data collected from business operations.

[0022] The expansion model is trained using the training data, a standard autoregressive cross-entropy loss function, and a label smoothing strategy, and sentences are generated.

[0023] The generated sentences are filtered by calculating the BLEU values ​​between sentences;

[0024] The filtered sentences are sorted using a preset sorting criterion.

[0025] According to a specific implementation of an embodiment of this disclosure, the method further includes:

[0026] By mixing a pre-defined list of proper nouns with the original training data to train the model, a corpus expansion model that preserves proper nouns is obtained.

[0027] According to one specific implementation of this disclosure, the preset number of seed corpora is 3-5.

[0028] Secondly, embodiments of this disclosure provide a deep learning-based text robot training system, the system comprising:

[0029] The data processing module is configured to organize a preset number of seed corpora based on the original intent, import the seed corpora into the robot knowledge base, identify problem data in all intents, and correct the problem data; wherein, all intents include the original intent and the seed corpora;

[0030] An expansion module is configured to batch expand all the intents to obtain expanded intents; identify whether the expanded intents contain obfuscated corpus; when obfuscated corpus is contained, the obfuscated corpus is corrected to obtain the expanded corrected intents;

[0031] The verification module is configured to train a text robot model using the original intent, the seed corpus, and the expanded and modified intent; verify whether the robot model meets expectations; wherein, when the robot model does not meet expectations, the robot model is retrained by expanding a single intent or adjusting or expanding a single corpus until the robot model meets expectations.

[0032] According to a specific implementation of an embodiment of this disclosure, the system further includes:

[0033] The expansion model training module is configured to clean, cluster, and resample the original data to construct training data; the original data is data collected from business operations; the expansion model is trained using the training data, a standard autoregressive cross-entropy loss function, and a label smoothing strategy to generate sentences; the generated sentences are filtered by calculating the BLEU values ​​between sentences; and the filtered sentences are ranked using a preset ranking metric.

[0034] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0035] At least one processor; and,

[0036] A memory communicatively connected to the at least one processor; wherein,

[0037] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the deep learning-based text robot training method described in any of the first aspects or any implementations thereof.

[0038] Fourthly, embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing computer instructions that, when executed by at least one processor, cause the at least one processor to perform the deep learning-based text robot training method in the first aspect or any implementation thereof.

[0039] Fifthly, embodiments of this disclosure also provide a computer program product, the computer program product including a computing program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to execute the deep learning-based text robot training method in the first aspect or any implementation thereof.

[0040] The deep learning-based text robot training method disclosed in this embodiment combines various deep learning algorithms, industry-accumulated data, and the robot's construction, optimization, and operation. Through steps such as batch corpus expansion (training deep learning models with industry-accumulated data), health checks (clustering and classification), and unknown question clustering (unsupervised machine learning and pre-trained similarity calculation), it simplifies robot operation and improves the robot's question-answering accuracy. It reduces the workload of traditional robot construction to the level of writing seed corpora (3-5 items), shortening the robot construction time from a week or even longer to 1-3 days. The health check automatically checks the health of the knowledge base, handles obfuscated intents and obfuscated corpora, and ensures the independence between various intents, making it more efficient and direct than the traditional method of manually asking questions and optimizing the robot line by line. Through unknown question clustering, it extracts knowledge not covered by the robot and recommends knowledge with high similarity to the knowledge base based on similarity calculation, increasing the speed of knowledge feedback and improving the knowledge base coverage. This multi-faceted approach ensures the robot's question-answering accuracy and is more efficient than the traditional method of manually screening conversation history to discover online problems. Attached Figure Description

[0041] The above is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] Figure 1 A schematic diagram of a deep learning-based text robot training method provided in this embodiment of the present disclosure;

[0043] Figure 2 A flowchart illustrating a deep learning-based text robot training method provided in this disclosure embodiment;

[0044] Figure 3 A schematic diagram of a deep learning-based text robot training system provided in this disclosure embodiment; and

[0045] Figure 4 A schematic diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0046] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0047] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0048] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0049] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0050] This invention provides a deep learning-based text robot training method. It involves pre-setting an initial intent (or standard question), then manually configuring several seed corpora for each intent. The seed corpora consist of several different ways of asking the same question. Based on these standard questions and seed corpora, more synonymous sentences with different expressions are automatically generated in batches (expanded corpora). The expanded corpora are then checked and corrected. Finally, the text robot model is trained using the initial intent (standard question), seed corpora, and expanded corpora. Through batch corpus expansion, health checks, and clustering of unknown questions, this method addresses the problem of conventional robot deployment in enterprises requiring significant effort from trainers with business and technical expertise.

[0051] Figure 1 This is a schematic diagram illustrating the process of a deep learning-based text robot training method provided in an embodiment of this disclosure.

[0052] Figure 2 To and Figure 1 The corresponding flowchart of the deep learning-based text robot training method.

[0053] like Figure 1 As shown, in step S110, a preset number of seed corpora are organized based on the original intent, and the seed corpora are imported into the text robot knowledge base.

[0054] More specifically, the robot trainer or client compiles 3 to 5 seed corpora for the original intent and imports them into the robot's (or text robot's) knowledge base via Excel.

[0055] More specifically, we now proceed to step S120.

[0056] In step S120, problem data in all intents is identified and the problem data is corrected; wherein, all intents include the original intent and the seed corpus.

[0057] In this embodiment of the invention, the question data in all intents includes at least one of: FAQ answer similarity data, confusion intent data, and confusion corpus data.

[0058] More specifically, the health check function identifies data in the original intent and seed corpus that are similar to frequently-asked questions (FAQs), obfuscated intents, and obfuscated corpora. By merging and splitting, problematic intents are corrected. That is, if the seed corpora under two intents are highly similar, the system will prompt and suggest that the implementer merge the two intents or make further modifications to the seed corpus to better distinguish between the two intents; if the answers corresponding to two intents are highly similar, the system suggests merging the two intents.

[0059] Next, proceed to step S130.

[0060] In step S130, all the intentions are batch expanded to obtain the expanded intentions.

[0061] The one-click expansion function allows for batch expansion of all intents, with a maximum of 750 entries for each FAQ and intent.

[0062] In this embodiment of the invention, the step of batch expanding all the intentions includes:

[0063] The original data is cleaned, clustered, and resampled to construct training data; the original data is data collected from business operations.

[0064] The expansion model is trained using the training data, a standard autoregressive cross-entropy loss function, and a label smoothing strategy, and sentences are generated.

[0065] The generated sentences are filtered by calculating the BLEU values ​​between sentences;

[0066] The filtered sentences are sorted using a preset sorting criterion.

[0067] In this embodiment of the invention, the method further includes:

[0068] By mixing a pre-defined list of proper nouns with the original training data to train the model, a corpus expansion model that preserves proper nouns is obtained.

[0069] More specifically, what we are doing here is corpus expansion, and the expansion capability mainly depends on the expansion model. The following is the process of obtaining the expansion model:

[0070] Given a sentence, the goal of paraphrasing is to generate paraphrased sentences that differ from the original sentence in vocabulary or syntax while preserving the original semantics. In recent years, with the rapid development of various deep neural network models, paraphrasing methods have gradually shifted from traditional methods to more advanced deep neural network approaches. Currently, most existing paraphrasing models are based on Seq2Seq models consisting of an encoder and a decoder. The encoder's main purpose is to extract semantic information, i.e., to obtain a vector representation of the contextual semantics of each word. Then, the decoder generates the paraphrased sentence based on the vectors provided by the encoder. Specifically, in each decoding step, the encoder's output vector is forward-propagated to obtain the distribution of words, from which appropriate words are sampled. Common decoding strategies include greedy decoding, beam search decoding, and kernel sampling decoding. By comparing existing open-source generative Seq2Seq pre-trained models MASS (Masked Sequence to Sequence Pre-training), T5 (Text-To-Text Transfer Transformer), and BART (Bidirectional and Auto-Regressive Transformers), we selected the BART model as the base model for fine-tuning, based on the different pre-training tasks.

[0071] The specific process for model fine-tuning and evaluation is as follows:

[0072] (1) Data preparation

[0073] Our goal is to take an original sentence as input and generate sentences with the same semantics but different expressions. This requires constructing a large number of similar sentence pairs as training data. The original data is collected from business operations and contains a large number of similar sentences. The training data is constructed by cleaning, clustering, and resampling the data.

[0074] Data cleaning: Filtering pure numeric text and some noisy data, and desensitizing text containing phone numbers, ID numbers, and email addresses.

[0075] Data clustering: Since the data is collected from different extended businesses, there are duplicate corpora and a large number of similar corpora between different businesses. First, duplicate corpora are filtered out by deduplication, and then similar corpora are integrated by density clustering to form the original similar corpus. Here, density clustering is implemented by calling the Fast Clustering method in the Sentence-Transformer framework.

[0076] Data Sampling: In the original similar corpus, the number of corpus words in different similar corpus clusters varies greatly. In order to ensure the generalization of the model, a sampling strategy is set to sample the data of different clusters separately, so as to make the number of similar sentence pairs composed of different clusters in the training data as even as possible, and obtain the final training data.

[0077] (2) Model Setup

[0078] During training, the standard autoregressive cross-entropy loss function was used. To improve the model's generalization ability and prevent overfitting, a label smoothing strategy was incorporated, where the smoothed label vector was used when calculating the loss. To replace the traditional ONT-HOT encoded label vector, let K be the total number of categories in the multi-class classification, α be a hyperparameter (here we take it as 0.05), i = target means that category i is the true target category, i ≠ target means that category i is not the true target category, i.e.

[0079]

[0080] In this way, the smoothed label distribution is equivalent to adding noise to the true distribution, which prevents the model from being overconfident in the correct label, making the difference between the predicted positive and negative sample output values ​​less large, thereby avoiding overfitting and improving the model's generalization ability.

[0081] During prediction generation, the model decodes based on Top_n and Top_p sampling strategies. Top_n refers to sampling probabilities from the top n words in the current time step's word probability distribution at each decoding step, using these sampled words as the output for that time step. Top_p refers to selecting a number of words with a sum of probabilities p in descending order of word probability in the distribution at each decoding step, and sampling them probably. In practice, different strategies can be used to improve the diversity of generated sentences, such as increasing the parameter values ​​for Top_n and Top_p sampling, setting temperature smoothing parameters, and enabling random dropout during decoding.

[0082] (3) Post-processing of generated corpus

[0083] Due to the randomness of decoding, there are sentences with high similarity in the same batch of generated sentences, that is, too many repeated words in the sentences, which does not meet the requirements of expression diversity. Therefore, we first filter the generated sentences, mainly by calculating the BLEU value between sentences. If the BLEU value is higher than a set threshold, one of them is deleted. In this way, we can obtain a batch of sentences with diverse expressions.

[0084] In practical applications, to enable customers to quickly select multiple available expanded sentences, a ranking metric was designed to sort the filtered sentences, placing higher-quality sentences at the top. The ranking metric is as follows:

[0085] sort_metric=a*similarity(x,yi)+b*BLEU,

[0086] BLEU=C*(0.1*bleu(x,yi))2*eT

[0087] Here, similarity(x,yi) is the semantic similarity between the generated sentence yi and the original sentence x, with a value ranging from [-1,1]; bleu(x,yi) is the geometric mean of the BLEU scores of the generated sentence yi and the original sentence x, specifically the 1-gram (i.e., accuracy of a single word) and the 2-gram (i.e., accuracy of two-word pairs), with a value ranging from [0,100], and the final BLEU score range is [0,1]; C and T are control parameters, mainly controlling the diversity of sentences in the preceding order after sorting; a and b are weight parameters, adjusting the importance of semantic similarity and sentence diversity to the sorting order. When calculating semantic similarity, the sentences are first encoded, and then the cosine similarity between sentences is calculated to obtain similarity(x,yi). The BLEU score is calculated using Python's sacrebleu package.

[0088] The above ranking metrics were designed primarily to meet clients' quality requirements for expanded sentences, namely, to obtain a batch of sentences that are semantically similar and expressively diverse. Semantic similarity is determined by calculating the similarity(x,yi) between the two sentences, with higher semantic similarity between the generated and original sentences considered better. Expressive diversity is measured by calculating the bleu(x,yi) between the two sentences, with bleu values ​​ranging from [0,100]. Generally, a higher bleu value indicates more repeated characters between the generated and original sentences, meaning lower expressive diversity. A lower bleu value increases the diversity of the generated sentences but also decreases semantic similarity. Therefore, ideally, high-quality corpora have bleu values ​​around the median of the range, ensuring both expressive diversity and high semantic similarity. By adjusting the parameters in the BLEU function, the generated sentences can be controlled to achieve the desired ideal effect.

[0089] To select appropriate parameters, we generate a batch of sentences for each given sentence, manually assess and label their usability, and then calculate the Spearman correlation coefficient between different parameter combinations and the manually labeled results to select a suitable set of parameters as the final ranking metric. Once the ranking metric is determined, specifying the number of sentences to generate ensures that the sentences retrieved after ranking all meet the quality requirements.

[0090] (4) Model Evaluation

[0091] To verify the effectiveness of the generative model, experiments were conducted on two implemented projects. The results showed that for each intent or standard question, when about three seed sentences could be provided by humans, the intent recognition model was directly trained using the corpus expanded by the generative model. Ultimately, the accuracy of intent recognition exceeded that of the model trained with manually configured corpus.

[0092] Experiments have demonstrated the usability of the corpus expansion model. In practical applications, implementers only need to write 3 to 5 seed sentences for each intent or standard question to expand a large amount of usable corpus as training data for the intent recognition model, thereby accelerating the project implementation.

[0093] (5) Corpus expansion model for proper noun retention

[0094] When decoding and generating sentences, the model generates them word by word. This can cause proper nouns in the original sentence to be unable to be directly decoded and generated. For example, in the original sentence "How do I apply for a driver's license for the village-to-village road?", "village-to-village road" is a proper noun. In the generated sentence, "village-to-village road" may not appear completely or may not convey the meaning. In order to ensure that proper nouns can be decoded completely, we adopted a hard copy strategy and retrained a new model.

[0095] 1) Data preparation

[0096] A proper noun list is summarized from the existing training dataset. The proper nouns in sentence pairs that contain both proper nouns are replaced with [special_token], that is, [special_token] appears in both the source and target sides. A portion of such training data is constructed and mixed with the original training data to train the model.

[0097] 2) Model Setup

[0098] During model training, a standard autoregressive loss function is used. During prediction generation, sentences without proper nouns are directly input into the model for decoding and generation. For sentences with proper nouns, the proper nouns in the sentence need to be replaced with [special_token] and input into the model for decoding and generation. Then, [special_token] in the generated sentence is replaced with the original proper noun. Finally, post-processing is performed on the generated sentences to obtain the corpus that meets the requirements.

[0099] Next, proceed to step S140.

[0100] In step S140, it is identified whether the expanded intent contains obfuscated material; if it contains obfuscated material, the obfuscated material is modified to obtain the expanded modified intent.

[0101] The health check function is used to check again whether there is any obfuscated corpus after the expansion. If there is no obfuscated corpus, proceed directly to step S150; if there is obfuscated corpus, the obfuscated corpus is corrected.

[0102] Next, proceed to step S150.

[0103] In step S150, the text robot model is trained using the original intent, the seed corpus, and the expanded and corrected intent.

[0104] More specifically, after correction, clicking the training button trains the text robot model using the original intent, the seed corpus, and the expanded and corrected intent.

[0105] Next, proceed to step S160.

[0106] In step S160, the text robot model is verified to meet expectations; wherein, when the text robot model does not meet expectations, the text robot model is retrained by expanding a single intent or adjusting or expanding a single corpus until the text robot model meets expectations.

[0107] More specifically, after training is completed, the robot model is verified to meet expectations through the debugging interface. That is, some questions are randomly input to test whether the robot model can successfully recognize the intent information contained in the user's input questions and reply with the corresponding answer. For intents that do not meet expectations, adjustments or expansions are made by expanding a single intent or a single corpus, and then step S120 is repeated.

[0108] In this embodiment of the invention, the method further includes:

[0109] The text robot model that meets the expected specifications will be released online.

[0110] The text robot model cannot recognize user questions, and hierarchical clustering is used to perform cluster analysis on the user questions to obtain the cluster analysis results.

[0111] Suggestions are provided based on the clustering analysis results; the suggestions include creating new intents based on the clustering analysis results or adding the clustering analysis results to the corpus of existing intents;

[0112] Whenever an intent is created or modified, problematic data in all intents is identified and corrected.

[0113] More specifically, after the test is passed, the robot model is retrained and deployed online. After deployment, questions that the robot cannot recognize from user queries are filtered out. Using the existing hierarchical clustering function, cluster analysis is performed on the unknown questions. Based on the clustering results, suggestions can be made to implementers to create new intents for each cluster or add the clusters to the existing intent corpus, according to the similarity between each cluster and the current intent cluster. This is used to supplement and improve the knowledge base. Step S140 is repeated to form a closed loop of intelligent operation.

[0114] Figure 3 The present invention illustrates a deep learning-based text robot training system 300, including a data processing module 310, an expansion module 320, and a verification module 330.

[0115] The data processing module 310 is used to organize a preset number of seed corpora based on the original intent, import the seed corpora into the robot knowledge base; identify problem data in all intents, and correct the problem data; wherein, all intents include the original intent and the seed corpora;

[0116] The expansion module 320 is used to batch expand all the intentions to obtain expanded intentions; identify whether the expanded intentions contain obfuscated corpus; when obfuscated corpus is contained, the obfuscated corpus is corrected to obtain the expanded corrected intentions;

[0117] The verification module 330 is used to train a text robot model using the original intent, the seed corpus, and the expanded and modified intent; and to verify whether the robot model meets expectations. When the robot model does not meet expectations, the robot model is retrained by expanding a single intent or adjusting or expanding a single corpus until the robot model meets expectations.

[0118] In this embodiment of the invention, the system further includes: an expansion model training module, configured to clean, cluster, and resample the original data to construct training data; the original data is data collected from business operations; the expansion model is trained using the training data, a standard autoregressive cross-entropy loss function, and a label smoothing strategy to generate sentences; the generated sentences are filtered by calculating the BLEU values ​​between sentences; and the filtered sentences are ranked using a preset ranking index.

[0119] See Figure 4 This disclosure also provides an electronic device 40, which includes:

[0120] At least one processor; and,

[0121] The memory is communicatively connected to the at least one processor; wherein,

[0122] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the deep learning-based text robot training method in the foregoing method embodiments.

[0123] This disclosure also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the deep learning-based text robot training method in the foregoing method embodiments.

[0124] This disclosure also provides a computer program product, which includes a computing program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions that, when executed by a computer, cause the computer to perform the deep learning-based text robot training method in the foregoing method embodiments.

[0125] The following is for reference. Figure 4 The diagram illustrates a structural schematic of an electronic device 40 suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0126] like Figure 4 As shown, electronic device 40 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of electronic device 40. The processing unit 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0127] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 40 to communicate wirelessly or wiredly with other devices to exchange data. Although electronic device 40 with various devices is shown in the figure, it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0128] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of embodiments of this disclosure.

[0129] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0130] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0131] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire at least two Internet Protocol (IP) addresses; send a node evaluation request including the at least two IP addresses to a node evaluation device, wherein the node evaluation device selects an IP address from the at least two IP addresses and returns it; and receive the IP address returned by the node evaluation device; wherein the acquired IP address indicates an edge node in a content delivery network.

[0132] Alternatively, the aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: receive a node evaluation request including at least two Internet Protocol (IP) addresses; select an IP address from the at least two IP addresses; and return the selected IP address; wherein the received IP address indicates an edge node in the content delivery network.

[0133] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0135] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0136] It should be understood that the various parts of this disclosure can be implemented in hardware, software, firmware, or a combination thereof.

[0137] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A deep learning-based text robot training method, characterized in that, The method includes the following steps: Based on the original intent, a preset number of seed corpora are organized, and the seed corpora are imported into the text robot knowledge base; Identify problematic data in all intents and correct the problematic data; wherein, all intents include the original intent and the seed corpus; Batch expand all the aforementioned intents to obtain the expanded intents; Identify whether the expanded intent contains obfuscated material; if it does, modify the obfuscated material to obtain the expanded modified intent; The text robot model is trained using the original intent, the seed corpus, and the expanded and corrected intent. Verify whether the text robot model meets expectations; wherein, when the text robot model does not meet expectations, retrain the text robot model by expanding a single intent or adjusting or expanding a single corpus until the text robot model meets expectations.

2. The deep learning-based text robot training method according to claim 1, characterized in that, The method further includes: The text robot model that meets the expected specifications will be released online. The text robot model cannot recognize user questions, and hierarchical clustering is used to perform cluster analysis on the user questions to obtain the cluster analysis results. Suggestions are provided based on the clustering analysis results; the suggestions include creating new intents based on the clustering analysis results or adding the clustering analysis results to the corpus of existing intents; Whenever an intent is created or modified, problematic data in all intents is identified and corrected.

3. The deep learning-based text robot training method according to claim 1 or 2, characterized in that, The question data in all intents includes at least one of: FAQ answer similarity data, misleading intent data, and misleading corpus data. 。 4. The deep learning-based text robot training method according to claim 1 or 2, characterized in that, The batch expansion of all the intents includes: The original data is cleaned, clustered, and resampled to construct training data; the original data is data collected from business operations. The expansion model is trained using the training data, a standard autoregressive cross-entropy loss function, and a label smoothing strategy, and sentences are generated. The generated sentences are filtered by calculating the BLEU values ​​between sentences; The filtered sentences are sorted using a preset sorting criterion.

5. The deep learning-based text robot training method according to claim 4, characterized in that, The method further includes: By mixing a pre-defined list of proper nouns with the original training data to train the model, a corpus expansion model that preserves proper nouns is obtained.

6. The deep learning-based text robot training method according to claim 1, characterized in that, The preset number of seed corpora is 3-5.

7. A deep learning-based text robot training system, characterized in that, The system includes: The data processing module is configured to organize a preset number of seed corpora based on the original intent, import the seed corpora into the robot knowledge base, identify problem data in all intents, and correct the problem data; wherein, all intents include the original intent and the seed corpora; An expansion module is configured to batch expand all the intents to obtain expanded intents; identify whether the expanded intents contain obfuscated corpus; when obfuscated corpus is contained, the obfuscated corpus is corrected to obtain the expanded corrected intents; The verification module is configured to train a text robot model using the original intent, the seed corpus, and the expanded and modified intent; verify whether the robot model meets expectations; wherein, when the robot model does not meet expectations, the robot model is retrained by expanding a single intent or adjusting or expanding a single corpus until the robot model meets expectations.

8. The deep learning-based text robot training system according to claim 7, characterized in that, The system also includes: The expansion model training module is configured to clean, cluster, and resample the original data to construct training data; the original data is data collected from business operations; the expansion model is trained using the training data, a standard autoregressive cross-entropy loss function, and a label smoothing strategy to generate sentences; the generated sentences are filtered by calculating the BLEU values ​​between sentences; and the filtered sentences are ranked using a preset ranking metric.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, cause the at least one processor to perform the deep learning-based text robot training method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that, when executed by at least one processor, cause the at least one processor to perform the deep learning-based text robot training method as described in any one of claims 1 to 6.

Images