Text processing method, device, server and storage medium

Abstract

The embodiment of the present application discloses a text processing method, device, server and storage medium. The embodiment of the present application can obtain the text to be processed. When the type of the text is subjective, the text is segmented according to a preset strategy to obtain the words that constitute the text; the surface similarity between the text and each corpus in the corpus is calculated based on the words; the information entropy similarity between the corpus with a surface similarity greater than a preset threshold and the text is calculated; and the corpus with the highest information entropy similarity is screened out. This scheme can calculate the surface similarity between the text and the corpus, and the surface similarity can be used to perform a preliminary screening of the corpus, as well as calculate the information entropy similarity between the corpus with a surface similarity greater than a preset threshold and the text, and screen out the corpus with the highest information entropy similarity, thereby improving the efficiency and accuracy of text processing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a text processing method, device, server and storage medium. Background Art

[0002] With the development of technology, many industries have pre-sales and after-sales services. For example, the express delivery industry provides multi-faceted services such as appointment, order checking, payment, and complaints. However, since manual customer service cannot be online 24 hours a day, and manual efficiency is low and cost is high, always-online intelligent customer service terminals have come into being.

[0003] At present, when answering questions, intelligent customer service terminals generally analyze the semantic information of the question first, and then search for relevant answers from the corpus content based on the semantic information. However, simple semantic matching makes it impossible to obtain some questions from the corpus, or the correlation between the searched answers and the questions is low, that is, the accuracy is low. Therefore, the answers provided by the intelligent customer service terminals cannot fully meet the needs of users. Moreover, it takes a lot of time to perform semantic matching, resulting in a significant lag in the responses of the intelligent customer service terminals. Summary of the invention

[0004] The embodiments of the present application provide a text processing method, device, server and storage medium, which can improve the efficiency and accuracy of text processing.

[0005] In a first aspect, an embodiment of the present application provides a text processing method, comprising:

[0006] Get the text to be processed;

[0007] When the type of the text is a subjective type, segmenting the text according to a preset strategy to obtain words constituting the text;

[0008] Calculating the surface similarity between the text and each piece of text in the corpus based on the words;

[0009] Calculate the information entropy similarity between the corpus whose surface similarity is greater than a preset threshold and the text;

[0010] Filter out the corpus with the highest information entropy similarity.

[0011] In some implementations, the calculating of the information entropy similarity between the corpus having a surface similarity greater than a preset threshold and the text includes:

[0012] extracting keywords from the words;

[0013] Filter out the corpus whose surface similarity is greater than the preset threshold to obtain the target corpus;

[0014] Calculating word information entropy between the keyword and the target corpus;

[0015] The information entropy similarity between the target corpus and the text is calculated based on the word information entropy.

[0016] In some implementations, calculating the word information entropy between the keyword and the target corpus includes:

[0017] Obtaining the total number of corpora in the corpus;

[0018] Determining the number of corpora in which the keyword appears in the corpus;

[0019] The word information entropy between the keyword and the target corpus is calculated according to the total number and the quantity.

[0020] In some implementations, calculating the surface similarity between the text and each piece of text in the corpus based on the words includes:

[0021] extracting keywords from the words;

[0022] Obtaining the number of words contained in each piece of the corpus that are identical to the keyword, and obtaining a first value;

[0023] Obtaining the number of words constituting the text obtained by word segmentation and the number of words obtained by word segmentation for each corpus, to obtain a second value;

[0024] The surface similarity between the text and each piece of text in the corpus is calculated according to the first value and the second value.

[0025] In some embodiments, the screening out the corpus with the highest information entropy similarity includes:

[0026] Scoring the corpus according to the information entropy similarity to obtain a score corresponding to the corpus;

[0027] Sorting the corpus in descending order of scores to obtain sorted corpus;

[0028] The corpus with the highest score is selected from the sorted corpus to obtain the corpus with the highest information entropy similarity.

[0029] In some implementations, after obtaining the text to be processed, the method further includes:

[0030] Acquire feature information contained in the text;

[0031] The type of the text is determined according to the feature information.

[0032] In some implementations, after determining the type of the text according to the feature information, the method further includes:

[0033] When the type of the text is an objective type, obtaining operation parameters contained in the text;

[0034] The corresponding result is calculated according to the operation parameters through a preset algorithm.

[0035] In some embodiments, the operation parameters include the starting address, the ending address and the weight of the items to be transported, the preset algorithm includes a freight algorithm, and the calculation of the corresponding results by the preset algorithm according to the operation parameters includes:

[0036] The corresponding freight result is calculated by the freight algorithm according to the starting address, the ending address and the weight of the items to be transported.

[0037] In some implementations, the operation parameters include a current date and an offset, and the calculation of a corresponding result using a preset algorithm according to the operation parameters includes:

[0038] The corresponding date result is calculated according to the current date and the offset using a preset algorithm.

[0039] In a second aspect, an embodiment of the present application further provides a text processing device, including:

[0040] A receiving module is used to obtain the text to be processed;

[0041] A word segmentation module, for segmenting the text according to a preset strategy to obtain words constituting the text when the type of the text is a subjective type;

[0042] A first calculation module, used for calculating the surface similarity between the text and each piece of corpus according to the words;

[0043] A second calculation module is used to calculate the information entropy similarity between the corpus whose surface similarity is greater than a preset threshold and the text;

[0044] The feedback module is used to filter out the corpus with the highest information entropy similarity.

[0045] In some embodiments, the second computing module includes:

[0046] An extraction unit, used to extract keywords from the words;

[0047] A screening unit is used to screen out corpora with surface similarity greater than a preset threshold to obtain target corpora;

[0048] A first calculation unit, used to calculate word information entropy between the keyword and the target corpus;

[0049] The second calculation unit is used to calculate the information entropy similarity between the target corpus and the text according to the word information entropy.

[0050] In some implementations, the first computing unit is specifically configured to:

[0051] Obtaining the total number of corpora in the corpus;

[0052] Determining the number of corpora in which the keyword appears in the corpus;

[0053] The word information entropy between the keyword and the target corpus is calculated according to the total number and the quantity.

[0054] In some implementations, the first computing module is specifically configured to:

[0055] extracting keywords from the words;

[0056] Obtaining the number of words contained in each piece of the corpus that are identical to the keyword, and obtaining a first value;

[0057] Obtaining the number of words constituting the text obtained by word segmentation and the number of words obtained by word segmentation for each corpus, to obtain a second value;

[0058] The surface similarity between the text and each piece of text in the corpus is calculated according to the first value and the second value.

[0059] In some embodiments, the feedback module is specifically used to:

[0060] Scoring the corpus according to the information entropy similarity to obtain a score corresponding to the corpus;

[0061] Sorting the corpus in descending order of scores to obtain sorted corpus;

[0062] The corpus with the highest score is selected from the sorted corpus to obtain the corpus with the highest information entropy similarity.

[0063] In some implementations, the text processing apparatus further includes:

[0064] The determination module is used to obtain feature information contained in the text and determine the type of the text according to the feature information.

[0065] In some implementations, the text processing apparatus further includes:

[0066] An acquisition module, used for acquiring operation parameters contained in the text when the type of the text is an objective type;

[0067] The third calculation module is used to calculate the corresponding result through a preset algorithm according to the operation parameters.

[0068] In some embodiments, the operation parameters include the starting address, the ending address and the weight of the items to be transported, the preset algorithm includes a freight algorithm, and the third calculation module is specifically used to: calculate the corresponding freight result through the freight algorithm according to the starting address, the ending address and the weight of the items to be transported.

[0069] In some implementations, the operation parameters include a current date and an offset, and the third calculation module is specifically configured to calculate a corresponding date result using a preset algorithm according to the current date and the offset.

[0070] In a third aspect, an embodiment of the present application further provides a server, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes any one of the text processing methods provided in the embodiments of the present application.

[0071] In a fourth aspect, an embodiment of the present application further provides a storage medium, wherein the storage medium is used to store a computer program, and the computer program is loaded by a processor to execute any text processing method provided in the embodiment of the present application.

[0072] The embodiment of the present application can obtain the text to be processed. When the type of the text is subjective, the text is segmented according to the preset strategy to obtain the words that make up the text; then the surface similarity between the text and each corpus in the corpus is calculated according to the words, and the information entropy similarity between the corpus with a surface similarity greater than a preset threshold and the text is calculated, and the corpus with the highest information entropy similarity can be screened out at this time. Since the scheme can calculate the surface similarity between the text and the corpus, the corpus with a surface similarity greater than a preset threshold can be first screened out from the corpus through the surface similarity, so as to achieve the initial screening of the corpus, and calculate the information entropy similarity between the corpus with a surface similarity greater than the preset threshold and the text, so that only the information entropy similarity between part of the corpus and the text can be calculated, so that the calculation speed can be improved, and then the corpus with the highest information entropy similarity can be screened out. The greater the information entropy similarity, the higher the similarity between the corpus and the text, so that the corpus corresponding to the text can be quickly and accurately screened based on the type of the text, thereby improving the efficiency and accuracy of text-based processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0074] Figure 1 It is a flowchart of a text processing method provided in an embodiment of the present application;

[0075] Figure 2 is another flowchart of the text processing method provided in an embodiment of the present application;

[0076] Figure 3 It is a flowchart of different types of problem handling methods provided by the embodiments of the present application;

[0077] Figure 4 is a schematic diagram of objective question calculation processing provided by an embodiment of the present application;

[0078] Figure 5 is a structural schematic diagram of a text processing device provided in an embodiment of the present application;

[0079] Figure 6 It is a schematic diagram of the structure of the server provided in the embodiment of the present application. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0081] See also Figure 1 , Figure 1 : is a flowchart of a text processing method provided by an embodiment of the present application. The execution subject of the text processing method may be a text processing device provided by an embodiment of the present application, or a server integrated with the text processing device, wherein the text processing device may be implemented in hardware or software. The text processing method may include:

[0082] S101, obtaining text to be processed.

[0083] For example, the text to be processed can be extracted from a text database, or text sent by a client can be received. The client can be a client on a terminal such as a mobile phone or a computer, and the text processing device can receive text sent by the user through a preset client on the terminal. The text can include words, letters, numbers, or punctuation marks, etc. The text can be a question initiated during the recruitment process, or a question initiated during the promotion process, or a question initiated during the homework process, or a question initiated during the task process, or a question initiated during the shopping process, or a question initiated during the chat, or a freight consultation question initiated by express delivery, or other types of text, the specific content of which is not limited here.

[0084] S102: When the type of the text is a subjective type, the text is segmented according to a preset strategy to obtain words constituting the text.

[0085] After receiving the text, the type of the text can be determined. For example, the type of the text can be divided into subjective type, objective type or other types, among which the subjective type can be a question-and-answer type, and the objective type can be a calculation type such as freight calculation or date calculation.

[0086] In some implementations, after obtaining the text to be processed, the text processing method may further include: obtaining feature information contained in the text; and determining the type of the text according to the feature information. The feature information may include keywords or semantic information.

[0087] When the type of the text is subjective, the text can be segmented according to a preset strategy to obtain the words that make up the text. The preset strategy can be flexibly set according to actual needs, and the specific content is not limited here.

[0088] For example, the preset strategy may be to segment words according to words in a dictionary or word library, for example, to segment "robot" into one word, to segment "patent" into one word, etc. The preset strategy may also be to segment the received text using the Jieba segmentation program package in the preset software.

[0089] After the text is segmented, one or more words that make up the text can be generated. For example, we can get the word set π(S) = {W1, W2, ..., W n}, where S represents text, W i Represents a word in a text, which can consist of one or more characters.

[0090] S103, calculating the surface similarity between the text and each piece of text in the corpus based on the words.

[0091] In some embodiments, calculating the surface similarity between a text and each piece of material in a corpus based on words may include: extracting keywords from the words; obtaining the number of words contained in each piece of material in the corpus that are identical to the keywords, and obtaining a first value; obtaining the number of words that constitute the text obtained by word segmentation, and the number of words obtained by word segmentation for each piece of material, and obtaining a second value; calculating the surface similarity between the text and each piece of material in the corpus based on the first value and the second value.

[0092] For example, keywords can be extracted from the words that make up the text. The keywords can be flexibly set according to actual needs. For example, the keywords can be obtained by removing stop words from the words obtained by word segmentation, and then the number of words contained in each piece of corpus that are the same as the keywords is obtained to obtain a first value, and the number of words that make up the text obtained by word segmentation and the number of words obtained by word segmentation for each piece of corpus are obtained to obtain a second value, which includes the number of words that make up the text obtained by word segmentation and the number of words obtained by word segmentation for each piece of corpus. Then, the surface similarity between the text and each piece of corpus can be calculated based on the first value and the second value. The calculation formula can be as follows:

[0093]

[0094] Among them, Sim s is the surface similarity, the text is S1, a corpus is S2, ∩ represents the intersection operation between sets, Γ represents the number of elements in the set, that is, the number of words in the corpus that are the same as the keywords, Len represents the length of the text or corpus, that is, the number of words obtained after the text or corpus is segmented, Len(S1) is the number of words that make up the text obtained by segmentation, Len(S2) is the number of words obtained by segmentation of the corpus, and the purpose of calculating the surface similarity is to filter out corpora with the same words from the corpus.

[0095] S104, calculating the information entropy similarity between the corpus whose surface similarity is greater than a preset threshold and the text.

[0096] In some embodiments, calculating the information entropy similarity between a corpus having a surface similarity greater than a preset threshold and a text may include: extracting keywords from words; screening out corpus having a surface similarity greater than a preset threshold to obtain a target corpus; calculating the word information entropy between the keywords and the target corpus; and calculating the information entropy similarity between the target corpus and the text based on the word information entropy.

[0097] For example, keywords can be extracted from the words that make up the text. The keywords can be flexibly set according to actual needs. Then, corpora with a surface similarity greater than a preset threshold are filtered out to obtain target corpora. At this time, the word information entropy between the keywords and the target corpora can be calculated, and the preset threshold can be flexibly set according to actual needs.

[0098] In some embodiments, calculating the word information entropy between keywords and target corpora may include: obtaining the total number of corpora in the corpus; determining the number of corpora in which the keywords appear in the corpus; and calculating the word information entropy between the keywords and the target corpora based on the total number and the number.

[0099] For example, the formula for calculating the word information entropy can be as follows:

[0100]

[0101] Among them, H represents the word information entropy, w represents the keyword, M represents the total number of corpora in the corpus, m represents the number of corpora in which the keyword w appears in the corpus. The larger the word information entropy value, the lower the frequency of the keyword appearing in the corpus. On the contrary, the smaller the word information entropy value, the higher the frequency of the keyword appearing in the corpus.

[0102] Then, the information entropy similarity between the target corpus and the text can be calculated based on the word information entropy. The calculation of the information entropy similarity between text S1 and target corpus S2 can be as follows:

[0103] Sim H =∑H(w i )

[0104] Among them, Sim H represents the information entropy similarity, w i ∈{π(S1)∩π(S2)}. The greater the information entropy similarity between text S1 and target corpus S2, the greater the similarity between text S1 and target corpus S2. By filtering out target corpora with a surface similarity greater than a preset threshold and calculating the information entropy similarity between the target corpus and the text, rather than calculating the information entropy similarity between corpora with a surface similarity less than or equal to the preset threshold and the text, the calculation speed can be improved. And through the calculation of the information entropy similarity, the effect of stop words (such as "de", "di", "huo", etc.) on the text can be effectively suppressed.

[0105] S105. Filter out the corpus with the highest information entropy similarity.

[0106] In some embodiments, screening out the corpus with the highest information entropy similarity may include: scoring the corpus according to the information entropy similarity to obtain a score corresponding to the corpus; sorting the corpus in order from high to low scores to obtain a sorted corpus; screening out the corpus with the highest score from the sorted corpus to obtain the corpus with the highest information entropy similarity.

[0107] Specifically, a mapping relationship between information entropy similarity and scores can be established, where one information entropy similarity corresponds to one score, or one information entropy similarity interval corresponds to one score, wherein the greater the information entropy similarity, the greater the score of the corresponding corpus, and vice versa, the smaller the information entropy similarity, the smaller the score of the corresponding corpus. For example, after obtaining the information entropy similarity, the information entropy similarity interval in which the information entropy similarity is located can be determined, and the score corresponding to the information entropy similarity can be determined based on the mapping relationship between the information entropy similarity interval and the score.

[0108] Then, the corpus can be sorted in order from high to low according to the score to obtain the sorted corpus. At this time, the corpus with the highest score can be screened out from the sorted corpus to obtain the corpus with the highest information entropy similarity. After obtaining the corpus with the highest information entropy similarity, the corpus can be fed back to the client according to actual needs, and displayed to the user by the client. In order to enrich the corpus in the corpus, when there is no corpus consistent with the text in the corpus, the text sent by the client can be stored in the corpus. Compared with machine learning algorithms such as deep learning, which require training of a large number of corpus samples, it will take a lot of computing resources and time, resulting in a large lag in feedback. The present application can calculate the information entropy similarity between the corpus and the text whose surface similarity is greater than a preset threshold, and screen out the corpus with the highest information entropy similarity, which can improve the calculation speed, and can also quickly and accurately give feedback to the client based on the type of text, thereby improving the efficiency and accuracy of text-based feedback.

[0109] In some embodiments, after determining the type of the text according to the feature information, the text processing method may further include: when the type of the text is an objective type, obtaining operation parameters contained in the text; and calculating corresponding results through a preset algorithm according to the operation parameters.

[0110] When the type of the text is an objective type, operation parameters can be extracted from the text, and the operation parameters can be flexibly set according to actual needs. For example, the operation parameters may include the starting address, the ending address, the weight of the items to be transported, the current date, and the offset of the logistics transportation. At this time, the corresponding results can be calculated by the preset algorithm according to the operation parameters. After obtaining the corresponding results, the results can be fed back to the client according to actual needs. The preset algorithm can be flexibly set according to actual needs. For example, the preset algorithm can include a freight algorithm or a date algorithm.

[0111] For example, when the text received from the client is: ninety-5, the calculation result of 90-5=85 can be calculated and fed back to the client. For another example, when the text received from the client is: ninety-two times eight, the calculation result of 92*8=736 can be calculated and fed back to the client.

[0112] In some embodiments, the operation parameters include the starting address, the ending address and the weight of the items to be transported, the preset algorithm includes a freight algorithm, and calculating the corresponding results through the preset algorithm according to the operation parameters may include: calculating the corresponding freight result through the freight algorithm according to the starting address, the ending address and the weight of the items to be transported.

[0113] For example, you can receive arithmetic problems related to freight sent by the client. When the text received from the client is: the consignment flow direction is from Beijing to Shenzhen, and the consignment weight is 3kg, the operation parameters that can be extracted from the text are: the starting address is Beijing, the ending address is Shenzhen, and the weight of the items to be transported is 3kg. Then, search the database to find that the first weight of the flow from Beijing to Shenzhen is 10 yuan, and the additional weight is 3 yuan. Therefore, the freight result can be calculated: 10+(3-1)*3=16. After obtaining the freight result, the freight result can be returned to the client according to actual needs.

[0114] In some embodiments, the operation parameters include the current date and the offset, and calculating the corresponding result by a preset algorithm according to the operation parameters may include: calculating the corresponding date result by a preset algorithm according to the current date and the offset. After obtaining the date result, the date result may be fed back to the client according to actual needs.

[0115] For example, when the text received from the client is: If you place an order now, when will the courier pick up the package? From this text, the operation parameters can be extracted as follows: the current date is 8:00 on July 6, and the time range for obtaining the door-to-door pickup is within 1 hour after the order is placed, that is, the offset is 1 hour. Then, the pickup time is calculated to be before 9:00 on July 6, so the date result can be obtained as before 9:00 on July 6, and the result is fed back to the client.

[0116] For another example, when the text received from the client is: When is the Mid-Autumn Festival this year? From the text, the operation parameters can be extracted as: In 2019, the Mid-Autumn Festival is the 15th day of the eighth lunar month, and then the 15th day of the eighth lunar month in 2019 is calculated to be September 13, so the result of September 13 can be fed back to the client.

[0117] For another example, when the text received from the client is: What year is this year? From the text, the operation parameter can be extracted as: This year is 2019, and then the zodiac sign of 2019 is calculated to be the Year of the Pig, so the result Year of the Pig can be fed back to the client.

[0118] For another example, when the text received from the client is: What year was last year? From the text, the operation parameter can be extracted as: last year was 2018, and then the zodiac sign of 2018 is calculated to be the year of the dog, so the result of the year of the dog can be fed back to the client.

[0119] The embodiment of the present application can obtain the text to be processed. When the type of the text is subjective, the text is segmented according to the preset strategy to obtain the words that make up the text; then the surface similarity between the text and each corpus in the corpus is calculated according to the words, and the information entropy similarity between the corpus with a surface similarity greater than a preset threshold and the text is calculated, and the corpus with the highest information entropy similarity can be screened out at this time. Since the scheme can calculate the surface similarity between the text and the corpus, the corpus with a surface similarity greater than a preset threshold (for example, the corpus with the same words) can be first screened out from the corpus through the surface similarity, and the corpus can be preliminarily screened, and the information entropy similarity between the corpus with a surface similarity greater than the preset threshold and the text can be calculated, so that only the information entropy similarity between part of the corpus and the text can be calculated, so that the calculation speed can be improved, and then the corpus with the highest information entropy similarity can be screened out. The greater the information entropy similarity, the higher the similarity between the corpus and the text, so that the client feedback can be given quickly and accurately based on the type of text according to actual needs, thereby improving the efficiency and accuracy of text processing.

[0120] The text processing method described in the above embodiment will be further described in detail below.

[0121] Please refer to Figure 2 , Figure 2 Another flowchart of the text processing method provided in the embodiment of the present application. The text processing method can be applied to the server, and the following will be described in detail by taking the example of receiving user inquiries on delivery time, door-to-door pickup time, freight and other related issues in the express delivery field, such as Figure 2 As shown, the process of the text processing method can be as follows:

[0122] S201. The server receives a question sent by the client.

[0123] The client may be a client on an intelligent customer service robot, and the server may receive questions about express delivery sent by the user through the client. The question may be one or more sentences, and the sentence may include one or more words, and the word may include one or more characters, and the character may include words, letters, numbers, or punctuation marks, etc.

[0124] S202: The server obtains characteristic information of the question.

[0125] The server can extract feature information from the question, which may include keywords or semantic information, etc. For example, when the received question is "freight from Shenzhen to Guangzhou", the extracted feature information may be freight, etc. For another example, when the received question is "Hello, my name is XXX, what is your name", the extracted feature information may be "you", "what", "name", etc.

[0126] S203: The server determines the type of the question based on the feature information.

[0127] For example, Figure 3 As shown, the types of questions can be divided into subjective questions and objective questions, among which the subjective type can be a question-and-answer type, and the objective type can be a calculation type such as freight calculation or date calculation. After obtaining the characteristic information, the type of question can be determined based on the characteristic information. For example, when the characteristic information of the question is freight, the question can be divided into an objective question, and when the characteristic information of the question is a name, the question can be divided into a subjective question.

[0128] S204: When the question type is a subjective question, the server segments the question and extracts keywords.

[0129] When the type of question is a subjective question, the server can segment the question according to the above preset strategy to obtain the words that make up the question, and extract keywords from the words that make up the question. The keywords can be flexibly set according to actual needs. For example, when the question is "the freight from Shenzhen to Guangzhou", the words obtained by segmentation can include "from", "Shenzhen", "to", "Guangzhou", "of", "freight", and the keywords extracted from them can include "Shenzhen", "to", "Guangzhou", "freight", etc.

[0130] S205: The server calculates the surface similarity between the question and each piece of text in the corpus based on the keywords.

[0131] For example, the server can obtain the number of words contained in each corpus of the corpus that are identical to the keywords, the number of words that constitute the question obtained by word segmentation, and the number of words obtained by word segmentation of each corpus. According to the number of words contained in the corpus that are identical to the keywords, the number of words that constitute the question, and the number of words obtained by word segmentation of the corpus, the surface similarity between the question and each corpus in the corpus is calculated according to the above-mentioned surface similarity calculation formula.

[0132] S206: The server selects the corpus whose surface similarity is greater than a preset threshold to obtain the target corpus.

[0133] For example, Figure 3 As shown, the server filter can remove the corpus with a surface similarity less than or equal to a preset threshold, and select the corpus with a surface similarity greater than the preset threshold to obtain the target corpus (i.e., candidate set), and the preset threshold can be flexibly set according to actual needs. The corpus with a surface similarity greater than the preset threshold is the corpus that has the same words as the received question, or the corpus that has more of the same words.

[0134] S207: The server calculates word information entropy between the keyword and the target corpus.

[0135] After obtaining the target corpus, the server can calculate the word information entropy between the keyword and the target corpus, for example, the total number of corpora in the corpus can be obtained. The number of corpora in which the keyword appears in the corpus is determined, and the word information entropy between the keyword and the target corpus is calculated according to the total number and the number according to the above word information entropy calculation formula.

[0136] S208. The server calculates the information entropy similarity between the target corpus and the question based on the word information entropy.

[0137] The server can calculate the information entropy similarity between the target corpus and the question according to the word information entropy according to the above information entropy similarity calculation formula. In this way, only the information entropy similarity between the target corpus and the question needs to be calculated, which improves the calculation speed.

[0138] S209: The server scores the corpus according to the information entropy similarity to obtain the score corresponding to the corpus.

[0139] The server can establish a mapping relationship between information entropy similarity intervals and scores. For example, after obtaining the information entropy similarity, the information entropy similarity interval in which the information entropy similarity is located can be determined, and the score corresponding to the information entropy similarity can be determined based on the mapping relationship between the information entropy similarity interval and the score.

[0140] Alternatively, the server may establish a mapping relationship between information entropy similarity and scores. For example, after obtaining the information entropy similarity, the server may determine the score corresponding to the information entropy similarity according to the mapping relationship between the information entropy similarity and the scores.

[0141] S210. The server feeds back the corpus with the highest score to the client.

[0142] For example, the server can sort the corpus in descending order of scores to obtain sorted corpus, which is included in the filtered target corpus, and then feed back the corpus with the highest score (i.e., the answer) to the client, such as Figure 3 As shown, the intelligent customer service robot corresponding to the client can display it to the user. In order to enrich the corpus, when there is no corpus consistent with the question in the corpus, the question sent by the client can be stored in the corpus.

[0143] S211. When the question type is an objective question, the server obtains the operation parameters contained in the question.

[0144] When the question type is an objective question, the server can extract operation parameters from the question, and the operation parameters can be flexibly set according to actual needs. For example, the operation parameters may include the starting address, ending address, weight of the items to be transported, current date, and offset of the logistics transportation.

[0145] S212: The server calculates corresponding results according to the operation parameters through a preset algorithm, and feeds the results back to the client.

[0146] The server calculates the corresponding result according to the operation parameters through a preset algorithm, and the preset algorithm can be flexibly set according to actual needs, for example, Figure 4 As shown, the calculation problem sent by the client can be received. When the problem received from the client is: the consignment flow direction is from Beijing to Shenzhen, the consignment weight is 3kg, how much is the freight? From this problem, the operation parameters can be extracted as follows: the starting address is Beijing, the end address is Shenzhen, and the weight of the item to be transported is 3kg. Then, the database is searched to find that the first weight of the flow from Beijing to Shenzhen is 10 yuan, and the additional weight is 3 yuan. Therefore, the freight result can be calculated: 10+(3-1)*3=16, and the freight result (i.e., the answer) is returned to the client, and displayed by the terminal such as the intelligent customer service robot corresponding to the client.

[0147] For example, Figure 3As shown, it is possible to receive arithmetic problems related to dates sent by the client. When the question received from the client is: If I place an order today, can the express delivery arrive the day after tomorrow? From this question, the operation parameters can be extracted as follows: the current date is July 6 (today), and the offset is 2 days (the day after tomorrow). Then, the date of the day after tomorrow is calculated to be July 8. Based on the fact that the express delivery can arrive in one day under normal circumstances, it is calculated that it can arrive on July 7, which is earlier than July 8. Therefore, the date result can be obtained as the day after tomorrow, and the feedback that it can arrive the day after tomorrow is given to the client.

[0148] For another example, when the client sends a question such as "When will the courier pick up the package if I place an order now?", the operation parameters extracted from the question are: the current date is 11:00 on September 6, and the time range for obtaining the door-to-door pickup is within 1 hour after the order is placed, that is, the offset is 1 hour. Then, the pickup time is calculated to be before 12:00 on September 6, so the date result can be obtained as before 12:00 on September 6, and the result is fed back to the client.

[0149] The embodiment of the present application can receive questions sent by the client. When the type of the question is subjective, the question can be segmented and keywords can be extracted; then the surface similarity between the question and each corpus in the corpus is calculated based on the keywords, and the word information entropy between the corpus with a surface similarity greater than a preset threshold and the question is calculated, and the information entropy similarity between the target corpus and the question is calculated based on the word information entropy. At this time, the corpus can be scored according to the information entropy similarity, and the corpus with the highest score (i.e., the answer) can be fed back to the client. When the type of the question is an objective question, the corresponding result (i.e., the answer) can be calculated based on the operation parameters contained in the question, and the result is fed back to the client. Since the scheme can divide the questions into objective questions and subjective questions based on the questions related to the delivery time, door-to-door collection time, and freight received from the user, after receiving the question, the type of the question can be judged, so that the question that the user needs to consult can be accurately located based on the question type, the search speed can be improved, and a reply can be given at a faster speed, and the answer with higher relevance can be replied to the user according to the type of the question, which improves the execution efficiency of the answer reply and ensures that the question and the answer maintain a high relevance. In addition, since the surface similarity, word information entropy, and information entropy similarity between questions and answers can be calculated to match answers to user questions, the execution efficiency is higher than that of machine learning models. It also supports multiple users to ask questions online concurrently, responds to user questions in a short time, and ensures a high correlation between questions and answers, saving manual customer service costs.

[0150] In order to better implement the text processing method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above text processing method. The meanings of the nouns are the same as those in the above text processing method, and the specific implementation details can refer to the description in the method embodiment.

[0151] See also Figure 5 , Figure 5 A structural diagram of a text processing device provided in an embodiment of the present application, wherein the text processing device 300 may include a receiving module 301, a word segmentation module 302, a first calculation module 303, a second calculation module 304 and a feedback module 305, etc.

[0152] The receiving module 301 is used to obtain the text to be processed.

[0153] The word segmentation module 302 is used to segment the text according to a preset strategy to obtain words constituting the text when the type of the text is subjective.

[0154] The first calculation module 303 is used to calculate the surface similarity between the text and each piece of corpus according to the words.

[0155] The second calculation module 304 is used to calculate the information entropy similarity between the corpus whose surface similarity is greater than a preset threshold and the text.

[0156] The feedback module 305 is used to select the corpus with the highest information entropy similarity.

[0157] In some implementations, the second calculation module 304 may include an extraction unit, a screening unit, a first calculation unit, a second calculation unit, etc., which may be specifically as follows:

[0158] An extraction unit, used to extract keywords from words;

[0159] A screening unit is used to screen out corpora with surface similarity greater than a preset threshold to obtain target corpora;

[0160] A first calculation unit, used to calculate word information entropy between keywords and target corpus;

[0161] The second calculation unit is used to calculate the information entropy similarity between the target corpus and the text according to the word information entropy.

[0162] In some implementations, the first calculation unit is specifically used to: obtain the total number of corpora in the corpus; determine the number of corpora in which the keyword appears in the corpus; and calculate the word information entropy between the keyword and the target corpus based on the total number and the number.

[0163] In some embodiments, the first calculation module 303 is specifically used to: extract keywords from words; obtain the number of words contained in each corpus of the corpus that are the same as the keywords, and obtain a first value; obtain the number of words that constitute the text obtained by word segmentation, and the number of words obtained by word segmentation for each corpus, and obtain a second value; calculate the surface similarity between the text and each corpus in the corpus based on the first value and the second value.

[0164] In some implementations, the feedback module 305 is specifically configured to:

[0165] Score the corpus according to the information entropy similarity and obtain the corresponding score of the corpus;

[0166] Sort the corpus in descending order of scores to obtain the sorted corpus;

[0167] The corpus with the highest score is selected from the sorted corpus to obtain the corpus with the highest information entropy similarity.

[0168] In some implementations, the text processing apparatus 300 may further include:

[0169] The determination module is used to obtain feature information contained in the text and determine the type of the text according to the feature information.

[0170] In some implementations, the text processing device 300 may further include an acquisition module and a third calculation module, etc., which may be specifically as follows:

[0171] An acquisition module, used for acquiring operation parameters contained in the text when the type of the text is an objective type;

[0172] The third calculation module is used to calculate the corresponding result through a preset algorithm according to the operation parameters.

[0173] In some embodiments, the operation parameters include the starting address, the ending address and the weight of the items to be transported, the preset algorithm includes a freight algorithm, and the third calculation module is specifically used to calculate the corresponding freight result through the freight algorithm based on the starting address, the ending address and the weight of the items to be transported.

[0174] In some implementations, the operation parameters include a current date and an offset, and the third calculation module is specifically configured to calculate a corresponding date result using a preset algorithm according to the current date and the offset.

[0175] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0176] In the embodiment of the present application, the receiving module 301 can obtain the text to be processed. When the type of the text is subjective, the word segmentation module 302 segments the text according to a preset strategy to obtain the words that constitute the text; then the first calculation module 303 calculates the surface similarity between the text and each piece of corpus in the corpus based on the words, and the second calculation module 304 calculates the information entropy similarity between the corpus whose surface similarity is greater than a preset threshold and the text. At this time, the feedback module 305 can screen out the corpus with the highest information entropy similarity. Since the scheme can calculate the surface similarity between text and corpus, the surface similarity can be used to first screen out corpus with the same words from the corpus, thereby performing a preliminary screening of the corpus, and calculating the information entropy similarity between the corpus and the text whose surface similarity is greater than a preset threshold. Thus, only the information entropy similarity between part of the corpus and the text can be calculated, which can increase the calculation speed, and then the corpus with the highest information entropy similarity can be screened out. The greater the information entropy similarity, the higher the similarity between the corpus and the text. Thus, feedback can be given quickly and accurately based on the type of text, thereby improving the efficiency and accuracy of text-based feedback.

[0177] The present application also provides a server, such as Figure 6 As shown, it shows a schematic diagram of the structure of the server involved in the embodiment of the present application, specifically:

[0178] The server may include one or more processing core processors 401, one or more computer-readable storage media memories 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will appreciate that Figure 6 The server structure shown in the figure does not constitute a limitation on the server, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0179] in:

[0180] The processor 401 is the control center of the server, and uses various interfaces and lines to connect various parts of the entire server. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 performs various functions of the server and processes data, thereby monitoring the server as a whole. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0181] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0182] The server also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions. The power supply 403 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0183] The server may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0184] Although not shown, the server may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the server will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby performing the following functions, as follows:

[0185] Obtain the text to be processed; when the type of the text is subjective, segment the text according to the preset strategy to obtain the words that make up the text; calculate the surface similarity between the text and each piece of corpus in the corpus based on the words; calculate the information entropy similarity between the corpus with a surface similarity greater than a preset threshold and the text; screen out the corpus with the highest information entropy similarity.

[0186] In some implementations, when calculating the information entropy similarity between a corpus having a surface similarity greater than a preset threshold and a text, the processor 401 may execute:

[0187] Extract keywords from words; filter out corpora with surface similarity greater than a preset threshold to obtain target corpora; calculate word information entropy between keywords and target corpora; calculate information entropy similarity between target corpora and text based on word information entropy.

[0188] In some embodiments, when calculating the word information entropy between the keyword and the target corpus, the processor 401 can execute: obtaining the total number of corpora in the corpus; determining the number of corpora in which the keyword appears in the corpus; and calculating the word information entropy between the keyword and the target corpus based on the total number and the number.

[0189] In some embodiments, when calculating the surface similarity between a text and each piece of material in a corpus based on words, the processor 401 may execute: extracting keywords from the words; obtaining the number of words contained in each piece of material in the corpus that are identical to the keywords, and obtaining a first value; obtaining the number of words that constitute the text obtained by word segmentation, and the number of words obtained by word segmentation for each piece of material, and obtaining a second value; and calculating the surface similarity between the text and each piece of material in the corpus based on the first value and the second value.

[0190] In some embodiments, when screening out the corpus with the highest information entropy similarity, the processor 401 can execute: scoring the corpus according to the information entropy similarity to obtain the score corresponding to the corpus; sorting the corpus in order from high to low scores to obtain sorted corpus; screening out the corpus with the highest score from the sorted corpus to obtain the corpus with the highest information entropy similarity.

[0191] In some implementations, after obtaining the text to be processed, the processor 401 may execute: obtaining feature information contained in the text; and determining the type of the text according to the feature information.

[0192] In some implementations, after determining the type of the text according to the feature information, the processor 401 may execute: when the type of the text is an objective type, obtaining operation parameters contained in the text; and calculating corresponding results through a preset algorithm according to the operation parameters.

[0193] In some embodiments, the operation parameters include the starting address, the ending address and the weight of the items to be transported, and the preset algorithm includes a freight algorithm. When the corresponding results are calculated by the preset algorithm according to the operation parameters, the processor 401 can execute: calculate the corresponding freight result by the freight algorithm according to the starting address, the ending address and the weight of the items to be transported.

[0194] In some implementations, the operation parameters include a current date and an offset. When calculating a corresponding result using a preset algorithm based on the operation parameters, the processor 401 may execute: calculating a corresponding date result using a preset algorithm based on the current date and the offset.

[0195] In the above embodiments, the description of each embodiment has its own focus. For the part that is not described in detail in a certain embodiment, please refer to the detailed description of the text processing method above, and will not be repeated here.

[0196] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0197] To this end, an embodiment of the present application provides a storage medium in which a computer program is stored, and the computer program can be loaded by a processor to execute any text processing method provided in the embodiment of the present application. For example, the computer program is loaded by the processor and can execute the following steps:

[0198] Obtain the text to be processed; when the type of the text is subjective, segment the text according to the preset strategy to obtain the words that make up the text; calculate the surface similarity between the text and each piece of corpus in the corpus based on the words; calculate the information entropy similarity between the corpus with a surface similarity greater than a preset threshold and the text; screen out the corpus with the highest information entropy similarity.

[0199] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0200] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0201] Since the instructions stored in the storage medium can execute any text processing method provided in the embodiments of the present application, the beneficial effects that can be achieved by any text processing method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0202] The above is a detailed introduction to a text processing method, device, server and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A text processing method, characterized in that: include: Get the text to be processed, including receiving the questions about express delivery issues sent by the client; Acquire feature information of the text, wherein the feature information includes keywords or semantic information; Determine the type of the text according to the feature information, the type including a subjective type or an objective type, the subjective type including a question-and-answer type, and the objective type including a calculation type; When the type of the text is a subjective type, segmenting the text according to a preset strategy to obtain words constituting the text; Calculating the surface similarity between the text and each piece of text in the corpus based on the words; Calculate the information entropy similarity between the corpus whose surface similarity is greater than a preset threshold and the text; Filter out the corpus with the highest information entropy similarity; When the type of the text is an objective type, obtaining operation parameters contained in the text; Calculate the corresponding result through a preset algorithm according to the operation parameters; Wherein, the operation parameters include the starting address, the ending address and the weight of the items to be transported, the preset algorithm includes a freight algorithm, and the corresponding result calculated by the preset algorithm according to the operation parameters includes calculating the corresponding freight result by the freight algorithm according to the starting address, the ending address and the weight of the items to be transported; or, The operation parameters include a current date and an offset, and calculating a corresponding result through a preset algorithm according to the operation parameters includes calculating a corresponding date result through a preset algorithm according to the current date and the offset.

2. The text processing method according to claim 1, characterized in that: The information entropy similarity between the corpus whose surface similarity is greater than a preset threshold and the text includes: extracting keywords from the words; Filter out the corpus whose surface similarity is greater than the preset threshold to obtain the target corpus; Calculating word information entropy between the keyword and the target corpus; The information entropy similarity between the target corpus and the text is calculated based on the word information entropy.

3. The text processing method according to claim 2, characterized in that: The calculating of word information entropy between the keyword and the target corpus includes: Obtaining the total number of corpora in the corpus; Determining the number of corpora in which the keyword appears in the corpus; The word information entropy between the keyword and the target corpus is calculated according to the total number and the quantity.

4. The text processing method according to claim 1, characterized in that: The calculating the surface similarity between the text and each piece of corpus according to the words comprises: extracting keywords from the words; Obtaining the number of words contained in each piece of the corpus that are identical to the keyword, and obtaining a first value; Obtaining the number of words constituting the text obtained by word segmentation and the number of words obtained by word segmentation for each corpus, to obtain a second value; The surface similarity between the text and each piece of text in the corpus is calculated according to the first value and the second value.

5. The text processing method according to claim 1, characterized in that: The corpus with the highest information entropy similarity screened out includes: Scoring the corpus according to the information entropy similarity to obtain a score corresponding to the corpus; Sorting the corpus in descending order of scores to obtain sorted corpus; The corpus with the highest score is selected from the sorted corpus to obtain the corpus with the highest information entropy similarity.

6. A text processing device, characterized in that: include: The receiving module is used to obtain the text to be processed, including receiving the express inquiry questions sent by the client; obtaining the feature information of the text, wherein the feature information includes keywords or semantic information; determining the type of the text according to the feature information, wherein the type includes a subjective type or an objective type, wherein the subjective type includes a question-and-answer type, and the objective type includes a calculation type; A word segmentation module, for segmenting the text according to a preset strategy to obtain words constituting the text when the type of the text is a subjective type; A first calculation module, used for calculating the surface similarity between the text and each piece of corpus according to the words; A second calculation module is used to calculate the information entropy similarity between the corpus whose surface similarity is greater than a preset threshold and the text; Feedback module, used to filter out the corpus with the highest information entropy similarity; The feedback module is also used to obtain the operation parameters contained in the text when the type of the text is an objective type; calculate the corresponding results through a preset algorithm according to the operation parameters; wherein the operation parameters include the starting address, the ending address and the weight of the items to be transported, and the preset algorithm includes a freight algorithm, and the corresponding results calculated through the preset algorithm according to the operation parameters include calculating the corresponding freight result through the freight algorithm according to the starting address, the ending address and the weight of the items to be transported; or, the operation parameters include the current date and an offset, and the corresponding results calculated through the preset algorithm according to the operation parameters include calculating the corresponding date result through the preset algorithm according to the current date and the offset.

7. A server, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the text processing method according to any one of claims 1 to 5 is executed.

8. A storage medium, characterized in that: The storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the text processing method according to any one of claims 1 to 5.

Images