Word segmentation optimization method and device, storage medium and computer equipment

Abstract

According to the word segmentation optimization method and device, the storage medium and the computer equipment provided by the invention, an initial customer service work order is obtained, and the initial customer service work order is preprocessed to obtain a target customer service work order; segmenting the target customer service work order to obtain multiple segments of work order texts; retrieving a fixed template corresponding to each section of work order text in a template library; and according to the template content and the template rule corresponding to each fixed template, extracting target text data from each section of work order text so as to perform word segmentation processing on each target text data. The interference of noise data is reduced by preprocessing the customer service work order; after the work order text is segmented, the work order text is matched with the fixed template in the template library, target text data is accurately extracted, interference of template content on word segmentation results is avoided, and therefore actual requirements of users are more accurately captured. Therefore, the accuracy of word segmentation statistics can be improved, and the platform can better reflect the real appeal of the user.

Description

Technical Field

[0001] The present application relates to the technical field of text processing, and in particular to a word segmentation optimization method, device, storage medium and computer equipment. Background Art

[0002] At present, when capturing the core demands of user feedback, public opinion platforms have adopted the method of automatically extracting problem descriptions and using word segmentation processing and word frequency statistics to strive to comprehensively and accurately present the focus of user concerns. However, in actual application, the fixed template content in customer service tickets often occupies a large proportion in the word segmentation statistics, which affects the effective extraction of users' real needs. Due to the universality and high frequency of these template contents, they have obscured the information focus of users' real feedback to a certain extent, causing the word segmentation results to deviate from users' actual concerns. This situation restricts the accuracy of the word segmentation statistics, making it difficult for the platform to fully and truly reflect the actual appearance of user problems, and reducing effective support for user demands. Summary of the invention

[0003] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defect that the fixed template content in the customer service work order in the prior art restricts the accuracy of the word segmentation statistics.

[0004] In a first aspect, the present application provides a word segmentation optimization method, the method comprising:

[0005] Obtain the initial customer service work order, and pre-process the initial customer service work order to obtain the target customer service work order;

[0006] Process the target customer service work order in sections to obtain multiple sections of the work order text;

[0007] In the template library, retrieve the fixed template corresponding to each work order text;

[0008] According to the template content and template rules corresponding to each fixed template, target text data is extracted from each section of the work order text to perform word segmentation processing on each target text data.

[0009] In one embodiment, the step of pre-processing the initial customer service work order to obtain the target customer service work order includes:

[0010] Identify the language type in the initial customer service ticket. If the initial customer service ticket includes multiple languages, unify the language in the initial customer service ticket to Chinese, and use the customer service ticket with only Chinese as the first customer service ticket.

[0011] Identify the front-end element symbols and special symbols in the first customer service work order, and remove each front-end element symbol and each special symbol to obtain a second customer service work order;

[0012] Punctuation marks in the second customer service work order are identified, and each predetermined punctuation mark in the second customer service work order is removed to obtain a target customer service work order, where the predetermined punctuation marks do not include a comma.

[0013] In one embodiment, the step of segmenting the target customer service work order to obtain multiple work order texts includes:

[0014] Split the target customer service ticket into multiple sections according to the position of the commas in the target customer service ticket.

[0015] In one embodiment, the step of retrieving a fixed template corresponding to each work order text in the template library includes:

[0016] Determine the text keyword set for each work order text and the template keyword set for each template in the template library;

[0017] The text keyword set of each work order text is matched with the template keyword set of each template in the template library, and the fixed template corresponding to each work order text is determined based on the matching result.

[0018] In one embodiment, the step of determining the fixed template corresponding to each work order text segment according to the matching result includes:

[0019] For each section of work order text, if the text keyword set of the section of work order text matches a consistent template keyword set, or matches a template keyword set with a similarity within a preset range, or matches a template keyword set with the same meaning, then the template containing the template keyword set is used as the fixed template for the section of work order text.

[0020] In one embodiment, the step of determining the fixed template corresponding to each work order text segment according to the matching result further includes:

[0021] For each work order text segment, if the work order text matches multiple template keyword sets, the keyword overlap between the text keyword set of the work order text and each matched template keyword set is calculated, and the template containing the template keyword set with the highest keyword overlap is used as the fixed template for the work order text segment.

[0022] In one embodiment, the step of extracting target text data from each section of work order text according to the template content and template rules corresponding to each fixed template includes:

[0023] For each section of work order text, if the template rule corresponding to the fixed template of the section of work order text is to delete the template and the content after it, then all the text content of the section of work order text will be deleted; if the template rule corresponding to the fixed template of the section of work order text is not to delete the content after the template, then regular matching is used to delete the text content in the section of work order text that corresponds to the template content of the fixed template to obtain the target text data of the section of work order text.

[0024] In a second aspect, the present application provides a word segmentation optimization device, the device comprising:

[0025] The customer service work order acquisition module is used to obtain the initial customer service work order and pre-process the initial customer service work order to obtain the target customer service work order;

[0026] The customer service work order segmentation module is used to segment the target customer service work order and obtain multiple sections of the work order text;

[0027] A fixed template retrieval module is used to retrieve the fixed template corresponding to each work order text in the template library;

[0028] The target text data extraction module is used to extract the target text data from each section of the work order text according to the template content and template rules corresponding to each fixed template, so as to perform word segmentation processing on each target text data.

[0029] In a third aspect, the present application provides a storage medium: the storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of any word segmentation optimization method in the above-mentioned embodiments.

[0030] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0031] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the steps of any word segmentation optimization method in the above embodiments are performed.

[0032] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0033] In the word segmentation optimization method, device, storage medium and computer equipment provided by this application, the customer service work order is preprocessed to reduce the interference of noise data and improve the clarity of the text; after the work order text is segmented, it is matched with the fixed template in the template library to accurately extract the target text data, avoid the interference of the template content on the word segmentation result, and more accurately capture the actual needs of users. This process improves the accuracy of word segmentation statistics, enables the platform to better reflect the real demands of users, optimizes public opinion analysis and feedback support capabilities, and ensures that the platform is more targeted and effective when dealing with user problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 A flowchart of a word segmentation optimization method provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of the structure of a word segmentation optimization device provided in an embodiment of the present application;

[0037] Figure 3 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0039] The present application provides a word segmentation optimization method. The following embodiments are described by taking the method applied to a computer device as an example. It can be understood that the computer device can be any device with data processing functions, including but not limited to a single server, a server cluster, a personal laptop computer, a desktop computer, etc. Figure 1 As shown, the method may include the following steps:

[0040] S101: Obtain an initial customer service work order, and pre-process the initial customer service work order to obtain a target customer service work order.

[0041] The initial customer service ticket refers to the original service request submitted by the user, which usually contains the questions, demands or feedback raised by the user. Preprocessing refers to the process of cleaning and standardizing the original data to make the data more standardized and easier to process. The target customer service ticket is the customer service ticket that has undergone preliminary preprocessing.

[0042] In this step, the customer service work order can be obtained through network request or file reading. The customer service work order can be stored in a database or stored as a text file. When the customer service work order needs to be processed, the original work order data can be extracted into the memory through API calls, database queries or file reading. Next, the initial customer service work order is preprocessed. Regular expressions can be used to match and remove irrelevant symbols in the work order, such as converting "Hello, world!!!" to "Helloworld", or the text can be formatted uniformly, such as unifying uppercase and lowercase, converting special characters or emoticons to standard characters, etc. After preprocessing, standardized text data for subsequent processing is obtained, that is, the target customer service work order.

[0043] S102: Process the target customer service work order in sections to obtain multiple sections of work order text.

[0044] Among them, segmentation is the process of dividing a long text into multiple smaller paragraphs or sub-text blocks. Multi-segment ticket text refers to multiple text fragments obtained through segmentation. Each fragment usually contains a part of the information in the customer service ticket and is a semantically complete paragraph or content block.

[0045] In this step, first, the segmentation rules are determined through text analysis, for example, text segmentation based on semantics, such as contextual keywords, sentence length, etc. Once the segmentation rules are determined, regular expressions, natural language processing toolkits, or custom text segmentation algorithms can be used to segment the target customer service tickets. After segmentation, each paragraph or subtext can be saved as a separate text block or data structure.

[0046] It is understandable that customer service tickets usually contain many different types of information, such as the user's problem description, customer service's response, terms of service, etc. If not segmented, this information may be mixed together, resulting in ambiguity or difficulty in distinguishing during analysis. By segmenting, the information in each paragraph can be clearly separated to avoid contextual confusion. After dividing the ticket text into multiple paragraphs, the content of each paragraph is shorter and the semantics are clear, so each paragraph can be processed more quickly, reducing the consumption of computing resources when processing large paragraphs of text and improving analysis efficiency.

[0047] S103: In the template library, a fixed template corresponding to each section of the work order text is retrieved.

[0048] The template library is a collection of predefined fixed templates. In the scenario of customer service tickets, the template library may store common system replies, standardized greetings, FAQ content, solutions to common problems, etc. Fixed templates refer to predefined text patterns stored in the template library, which usually have fixed formats or contents and are applicable to different user tickets. For example, common greetings or standardized answers to common questions.

[0049] In this step, you first need to load the template library from a database, file, or other storage medium. Next, compare each work order text with the fixed templates in the template library, for example, by using vocabulary matching, sentence structure analysis, or using a natural language processing model to understand the semantics of the paragraph. When a match is found, extract the corresponding fixed template, and mark or associate these templates with the relevant parts of the work order text. If the work order text contains multiple paragraphs or multiple templates, retrieve the corresponding fixed template for each paragraph.

[0050] It can be understood that with the pre-prepared template library, there is no need to fully analyze each work order text. Instead, by quickly matching the corresponding template, common fixed content can be directly associated with the text in the work order, which greatly speeds up the text processing process and avoids repetitive content from being processed again.

[0051] S104: extracting target text data from each section of the work order text according to the template content and template rules corresponding to each fixed template, so as to perform word segmentation processing on each target text data.

[0052] Among them, template content refers to the specific text content associated with a fixed template, which can be fixed words in common customer service tickets, such as customer demands, customer problems, member problems and demands, product names, etc. Template rules refer to the specific strategies defined in the template library for matching and processing fixed templates in the ticket text, which determine how to apply the template to the text content in the ticket, and are used to clarify how to process the template content, so as to extract useful user feedback information. Target text data refers to the actual data extracted from the ticket text that conforms to the template rules. This data is usually the core information of user feedback in the ticket, such as specific problem descriptions, requirements or demands. Word segmentation is a technology in natural language processing that aims to cut a continuous text into separate words or phrases.

[0053] In this step, firstly, each paragraph in the work order text is scanned by algorithms, such as regular expressions, keyword matching, grammar-based or semantic-based models, to identify whether it contains known fixed templates. This can be done by scanning keywords, sentence patterns or syntactic structures in the work order text. After that, decide how to process the matched template content according to the pre-defined template rules. For example, mark the template content, replace specific placeholders in the template, and process the template content in layers. Once the target text is determined, extract the core information from the matching paragraphs, such as problem descriptions, service requests, etc. Finally, word segmentation can be performed based on the extracted target text data.

[0054] Furthermore, if the work order text does not match the corresponding fixed template, it means that the work order text does not use fixed words, and the work order text can be directly used as the target text data.

[0055] It is understandable that customer service tickets often contain template content or standard replies, which are not helpful for analyzing user problems. By extracting target text data, it is possible to effectively filter out real user feedback from customer service tickets and remove irrelevant standard replies or redundant parts. Template rules can ensure that relevant template content is matched to obtain text data that is closely related to user needs, thereby reducing the possibility of erroneously extracting irrelevant information and improving the accuracy of data extraction.

[0056] In the above embodiment, by preprocessing the customer service work order, the interference of noise data is reduced and the clarity of the text is improved; after the work order text is segmented, it is matched with the fixed template in the template library to accurately extract the target text data, avoiding the interference of the template content on the word segmentation results, thereby more accurately capturing the actual needs of users. This process improves the accuracy of word segmentation statistics, enables the platform to better reflect the real demands of users, optimizes public opinion analysis and feedback support capabilities, and ensures that the platform is more targeted and effective when handling user issues.

[0057] In one embodiment, the step of preprocessing the initial customer service work order to obtain the target customer service work order includes:

[0058] Identify the language type in the initial customer service ticket. If the initial customer service ticket includes multiple languages, unify the language in the initial customer service ticket to Chinese, and use the customer service ticket with only Chinese as the first customer service ticket.

[0059] Identify the front-end element symbols and special symbols in the first customer service work order, and remove each front-end element symbol and each special symbol to obtain a second customer service work order;

[0060] Punctuation marks in the second customer service work order are identified, and each predetermined punctuation mark in the second customer service work order is removed to obtain a target customer service work order, where the predetermined punctuation marks do not include a comma.

[0061] Among them, language type refers to the language used in the text. For example, the work order may contain different languages ​​such as Chinese, English, and French. Identifying the language type refers to determining the language in the text. The first customer service work order is the work order after the multiple languages ​​in the initial customer service work order are unified into Chinese. By converting multilingual content into Chinese, the consistency of subsequent processing is ensured to avoid inconsistent processing caused by language differences. Among them, if the language type identified by the initial customer service work order is only Chinese, then the initial customer service work order is also processed as the first customer service work order. Front-end element symbols are usually related to web pages or user interfaces, and may include HTML tags, CSS style tags, script tags, etc. For example, ` `、` `、` `, etc. Special symbols refer to characters other than regular punctuation marks, such as mathematical symbols, currency symbols, quotation marks, brackets, etc. The second customer service ticket is a ticket that has been stripped of front-end element symbols and special symbols. It usually refers to the remaining plain text ticket after removing characters related to page presentation. The target customer service ticket refers to the final ticket obtained after removing specific punctuation marks. Predetermined punctuation marks refer to non-text symbols, such as periods, question marks, exclamation marks, etc. All predetermined punctuation marks must be removed, and the predetermined punctuation marks do not include commas.

[0062] Specifically, you can first determine the language of the text through natural language processing tools or language recognition libraries. The recognition process is based on analyzing features such as words, grammatical structures, and word frequency distribution in the text, and returns the type of language used in the text. If the text contains multiple languages, you can use rules or models to perform language detection to determine which parts are Chinese and which parts are other languages. Once it is detected that the work order contains multiple languages, you can use machine translation services to translate the non-Chinese parts into Chinese. Finally, a pure Chinese work order is generated to ensure language consistency.

[0063] After that, you can use regular expressions or predefined rule sets to identify front-end element symbols and special symbols in the ticket text. For front-end element symbols, you can use regular expressions or parsers to remove all front-end element symbols. For special symbols, you can use character set detection algorithms to identify and remove symbols. Next, you can identify all punctuation symbols in the ticket based on a predefined punctuation list. Among them, you can use regular expressions or character replacement functions to remove these punctuation symbols, but keep commas.

[0064] In this embodiment, multiple languages ​​are unified into Chinese to ensure the uniformity of subsequent processing. Texts in different languages ​​may use different grammatical structures, vocabularies and expressions. Unifying the language can eliminate the processing complexity caused by language differences. Front-end element symbols and special symbols are usually irrelevant to the core content of user feedback. They may be HTML tags, formatting symbols, control characters, etc. Removing these contents can make the text cleaner, and the remaining text content will be more in line with the information of actual user feedback, reduce irrelevant interference, and avoid affecting subsequent tasks such as sentiment analysis and keyword extraction. Punctuation is a common element in language, but for word segmentation, punctuation may interfere with the accuracy of information extraction. Removing punctuation helps to convert text into a more standard processing format.

[0065] In one embodiment, the step of segmenting the target customer service work order to obtain multiple work order texts includes:

[0066] Split the target customer service ticket into multiple sections according to the position of the commas in the target customer service ticket.

[0067] Specifically, a text processing tool can be used to identify all comma positions in the text. For example, a regular expression can match all comma symbols and record their index positions. Based on the position of the comma, the target work order text can be cut into multiple substrings. Each substring represents the text from one comma position to the next comma position. Among them, the string can be split according to the index of the comma by using a text cutting method. In this way, multiple sections of work order text can be obtained.

[0068] In this embodiment, commas usually indicate pauses in Chinese sentences and are often used to separate parallel components, additional information or short sentences in a sentence. Therefore, using commas as the basis for text segmentation helps to decompose long or complex sentences into multiple smaller, more logically independent parts, so that different meanings in the text can be better understood and processed, avoiding confusion or information loss that may be caused by processing long sentences at one time. At the same time, the sentences are smallest after comma segmentation, which is also convenient for template setting and matching.

[0069] In one embodiment, the step of retrieving a fixed template corresponding to each work order text in the template library includes:

[0070] Determine the text keyword set for each work order text and the template keyword set for each template in the template library;

[0071] The text keyword set of each work order text is matched with the template keyword set of each template in the template library, and the fixed template corresponding to each work order text is determined based on the matching result.

[0072] The text keyword set of each work order text refers to the most representative set of keywords extracted from each work order paragraph. These keywords are words that reflect the core information of the work order paragraph. The text keyword set usually includes keywords for problem description, customer needs, and feedback content. The template keyword set refers to the set of keywords predefined in each template. These keywords represent the core content or problem type of the template.

[0073] Specifically, keywords are first extracted from each work order paragraph, usually using natural language processing tools such as Jieba word segmentation, SpaCy or TF-IDF algorithm for keyword extraction. Through these methods, the words or phrases that best represent the core meaning of the work order paragraph can be identified. For example, the keywords "query" and "order status" are extracted from "I want to check my order status". The keyword extraction process can also use technologies such as part-of-speech tagging and named entity recognition to improve accuracy, thereby identifying more semantic information such as "order number" and "query".

[0074] Each template in the template library will also have a set of keywords, which are usually predefined manually and reflect the scenario or problem type represented by the template. For example, a "query order status" template may contain keywords such as "query", "order status", "order number", etc. These keywords can be extracted through manual maintenance or machine learning, and statistical analysis can be performed based on historical work order data to find the keyword set related to a specific template.

[0075] The keyword set of each work order text is matched with the keyword set in the template library. The similarity between two keyword sets can be determined by calculating text similarity, such as using cosine similarity, Jaccard similarity, etc. In one example, once the matching degree exceeds a certain set threshold, the system considers that the work order text matches the corresponding template. For example, if "query" and "order status" in the work order text successfully match the keyword set of the "query" and "order status" templates in the template library, the work order paragraph is classified as the "query order status" template.

[0076] In this embodiment, the template keyword set can efficiently identify and classify different types of customer questions. By comparing the keyword set of the text and the keyword set of the template, the specific category of the work order text can be simply and efficiently determined.

[0077] In one embodiment, the step of determining the fixed template corresponding to each work order text segment according to the matching result includes:

[0078] For each section of work order text, if the text keyword set of the section of work order text matches a consistent template keyword set, or matches a template keyword set with a similarity within a preset range, or matches a template keyword set with the same meaning, then the template containing the template keyword set is used as the fixed template for the section of work order text.

[0079] Specifically, for each section of work order text, a keyword set of the text is extracted through a keyword extraction algorithm. The keyword set of each template has been predefined in the template library, and the keyword set of each template can be obtained by querying the database or directly accessing the file storing the template. The keyword set of each section of the work order text is compared with the template keyword set. First, check whether there is a completely consistent keyword set. For the case of incomplete consistency, the method of calculating text similarity can be used to evaluate the similarity between keyword sets. If the similarity exceeds the preset threshold, the two sets are considered to match. For the case of the same meaning but different keywords, a word meaning matching algorithm can be used to determine whether there is a semantic similarity, and then determine whether it matches. If the matching degree between the text keyword set and a template keyword set reaches the preset standard, that is, completely consistent, highly similar or synonymous, then the template is the fixed template for the section of the work order text.

[0080] In this embodiment, by setting multiple matching conditions, such as exact match, similarity match and meaning match, a variety of language expressions can be processed, ensuring that even if there are grammatical variations, vocabulary replacements or synonym substitutions, the appropriate template can be successfully matched, thereby improving flexibility and accuracy and reducing matching failures or omissions caused by differences in user expressions.

[0081] In one embodiment, the step of determining the fixed template corresponding to each work order text segment according to the matching result further includes:

[0082] For each work order text segment, if the work order text matches multiple template keyword sets, the keyword overlap between the text keyword set of the work order text and each matched template keyword set is calculated, and the template containing the template keyword set with the highest keyword overlap is used as the fixed template for the work order text segment.

[0083] Specifically, if the text keyword set of each work order text segment is compared with the keyword sets of all templates in the template library, multiple templates matching the text keyword set are found, then for each matching template, the overlap between the keyword set of the work order text and the keyword set of the template is calculated, usually the ratio of the size of the intersection of the two keyword sets to the union is calculated, or the word frequency statistics are used to measure the overlap. After calculating the overlap of each matching template, the template keyword set with the highest overlap is selected as the final fixed template for the work order text segment.

[0084] In this embodiment, in the case of multiple template matching, the keyword sets between the work order text and different templates may have different degrees of similarity. Calculating the keyword overlap helps to quantify this similarity, so as to select the most relevant template from multiple matching templates, avoiding the problem of template inaccuracy caused by matching multiple templates.

[0085] In one embodiment, the step of extracting target text data from each section of work order text according to the template content and template rules corresponding to each fixed template includes:

[0086] For each section of work order text, if the template rule corresponding to the fixed template of the section of work order text is to delete the template and the content after it, then all the text content of the section of work order text will be deleted; if the template rule corresponding to the fixed template of the section of work order text is not to delete the content after the template, then regular matching is used to delete the text content in the section of work order text that corresponds to the template content of the fixed template to obtain the target text data of the section of work order text.

[0087] Specifically, first, the next operation is determined based on the template rules of the fixed template corresponding to each section of the work order text. For fixed templates that meet the "delete the template and the content after" rule, the template content in the work order text and all the text after the template need to be deleted. You can find the position of the template content in the text, such as string matching or regular expressions, to find the starting position of the template content, and then delete that position and all the content after it. For fixed templates that meet the "do not delete the content after the template" rule, you only need to use regular expressions to match the part of the work order text that matches the template content, find and delete this part of the content. After deletion, the remaining part, that is, the "content after the template", remains unchanged. After the deletion operation is completed, the remaining text part is the "target text data" of the section of the work order text, which is the cleaned text after removing irrelevant information.

[0088] Among them, regular expressions can quickly locate the part of the text that matches the fixed template. The working principle of regular matching is to extract the content that matches the pattern from the work order text and delete it by defining the pattern of the template.

[0089] In this embodiment, by judging the template rules, it is possible to automatically decide how to process the work order text. Template rules are defined in advance, and different processing operations are performed for different types of templates to ensure that each section of the work order text follows the corresponding rules when processing, thereby improving the accuracy and consistency of automated processing. In some cases, the template content and the text following it in the work order are no longer needed. Deleting these irrelevant parts can reduce the interference of irrelevant information, and the work order text becomes more concise and clear. For the rule that meets the "do not delete the content after the template", only the part that matches the template content is deleted, and the parts that are repeated or irrelevant to the template are removed. Useful information is retained so that subsequent analysis can be more accurate and unnecessary complexity can be reduced.

[0090] To facilitate understanding of the solution of the present application, specific examples are provided below for illustration.

[0091] Assume there are multiple customer service tickets, such as "Customer issue: Order not shipped, customer calls to inquire about order shipping issues Customer demand: urge for quick delivery Customer Service: I understand your need to use the product. I also hope that the product can be delivered to you as soon as possible. I have urged the warehouse to arrange it as soon as possible. ”.

[0092] For the template rule of "Do not delete the content after the template", the current processing is "Customer problem: order has not been shipped", and the created templates are: customer appeal, customer problem, member problem and appeal, product name, etc., then traverse these templates and find the template that matches the customer problem, then the text changes from "Customer problem: order has not been shipped" to "Order has not been shipped", and then continue to process "Customer calls online to inquire about order shipment issues". There is no text that is exactly the same as in the template, so this sentence is retained. Assuming that there are only these two sentences, the final sentence used for word segmentation is combined into: Order has not been shipped and customer calls online to inquire about order shipment issues. This kind of word segmentation is entirely a user problem, without interference from other sentences.

[0093] For the template rule of "Delete the template and the content after the template", after matching, not only the text that is the same as the template is removed, but also the entire text is discarded. For example, "Customer service plan: I understand your feelings about needing to use the product. I also hope that the product can be delivered to you as soon as possible. I have urged the warehouse to arrange it as soon as possible for you." If there is a customer service plan in the template configuration, this paragraph will be deleted.

[0094] The following describes the word segmentation optimization device provided in the embodiment of the present application. The word segmentation optimization device described below and the word segmentation optimization method described above can be referred to in correspondence with each other. Figure 2 As shown, the present application provides a word segmentation optimization device, the device comprising:

[0095] The customer service work order acquisition module 201 is used to acquire an initial customer service work order and pre-process the initial customer service work order to obtain a target customer service work order;

[0096] The customer service work order segmentation module 202 is used to segment the target customer service work order to obtain multiple work order texts;

[0097] The fixed template search module 203 is used to search the fixed template corresponding to each work order text in the template library;

[0098] The target text data extraction module 204 is used to extract target text data from each section of the work order text according to the template content and template rules corresponding to each fixed template, so as to perform word segmentation processing on each target text data.

[0099] In one embodiment, the customer service work order acquisition module 201 includes:

[0100] A first customer service work order determination unit, configured to identify a language type in an initial customer service work order. If the initial customer service work order includes multiple language types, the language in the initial customer service work order is unified into Chinese, and the customer service work order whose language type is only Chinese is used as the first customer service work order;

[0101] A second customer service work order determining unit, configured to identify front-end element symbols and special symbols in the first customer service work order, and remove each front-end element symbol and each special symbol to obtain a second customer service work order;

[0102] The target customer service work order determination unit is used to identify punctuation marks in the second customer service work order and remove each predetermined punctuation mark in the second customer service work order to obtain the target customer service work order, and the predetermined punctuation marks do not include commas.

[0103] In one embodiment, the customer service ticket segmentation module 202 includes:

[0104] The customer service ticket segmentation unit is used to split the target customer service ticket into multiple ticket text segments according to the position of commas in the target customer service ticket.

[0105] In one embodiment, the fixed template retrieval module 203 includes:

[0106] A keyword set determination unit, used to determine a text keyword set for each work order text segment and a template keyword set for each template in the template library;

[0107] The fixed template retrieval unit is used to match the text keyword set of each work order text with the template keyword set of each template in the template library, and determine the fixed template corresponding to each work order text according to the matching result.

[0108] In one embodiment, the fixed template retrieval unit includes:

[0109] The first fixed template retrieval subunit is used for, for each section of work order text, if the text keyword set of the section of work order text matches a consistent template keyword set, or matches a template keyword set with a similarity within a preset range, or matches a template keyword set with the same meaning, then the template containing the template keyword set is used as the fixed template for the section of work order text.

[0110] In one embodiment, the fixed template retrieval unit further includes:

[0111] The second fixed template retrieval subunit is used to calculate the keyword overlap between the text keyword set of the work order text and each matched template keyword set for each section of work order text, if the work order text matches multiple template keyword sets, and use the template containing the template keyword set with the highest keyword overlap as the fixed template for the section of work order text.

[0112] In one embodiment, the target text data extraction module 204 includes:

[0113] The target text data extraction unit is used to, for each section of work order text, if the template rule corresponding to the fixed template of the section of work order text is to delete the template and the content after it, then delete all the text content of the section of work order text; if the template rule corresponding to the fixed template of the section of work order text is not to delete the content after the template, then use regular matching to delete the text content in the section of work order text that corresponds to the template content of the fixed template, so as to obtain the target text data of the section of work order text.

[0114] In one embodiment, the present application also provides a storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the word segmentation optimization method described in any of the above embodiments.

[0115] In one embodiment, the present application further provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the word segmentation optimization method as described in any one of the above embodiments.

[0116] Indicatively, Figure 3 As shown, Figure 3 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 may be provided as a server. Figure 3 The computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by a memory 301, for storing instructions that can be executed by the processing component 302, such as an application. The application stored in the memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the word segmentation optimization method of any of the above embodiments.

[0117] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0118] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0119] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, "one", "one", "said", "the" and "it" may also include plural forms, unless the context clearly indicates another way. A plurality refers to at least two cases, such as 2, 3, 5 or 8, etc. "And / or" includes any and all combinations of the relevant listed items.

[0120] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0121] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A word segmentation optimization method, characterized in that: The method comprises: Obtain an initial customer service work order, and pre-process the initial customer service work order to obtain a target customer service work order; Processing the target customer service work order in sections to obtain multiple sections of work order text; In the template library, a fixed template corresponding to each section of the work order text is retrieved; According to the template content and template rules corresponding to each of the fixed templates, target text data is extracted from each section of the work order text to perform word segmentation processing on each of the target text data.

2. The word segmentation optimization method according to claim 1, characterized in that: The step of preprocessing the initial customer service work order to obtain a target customer service work order includes: Identify the language type in the initial customer service work order. If the initial customer service work order includes multiple language types, unify the language in the initial customer service work order into Chinese, and use the customer service work order whose language type is only Chinese as the first customer service work order; Identify the front-end element symbols and special symbols in the first customer service work order, and remove each of the front-end element symbols and each of the special symbols to obtain a second customer service work order; Punctuation marks in the second customer service work order are identified, and each predetermined punctuation mark in the second customer service work order is removed to obtain the target customer service work order, wherein the predetermined punctuation mark does not include a comma.

3. The word segmentation optimization method according to claim 2, characterized in that: The step of processing the target customer service work order in sections to obtain multiple sections of work order text includes: According to the positions of commas in the target customer service work order, the target customer service work order is divided into multiple sections of work order text.

4. The word segmentation optimization method according to claim 1, characterized in that: The step of retrieving the fixed template corresponding to each section of the work order text in the template library includes: Determining a text keyword set for each section of the work order text and a template keyword set for each template in the template library; The text keyword set of each section of the work order text is matched with the template keyword set of each template in the template library, and the fixed template corresponding to each section of the work order text is determined according to the matching result.

5. The word segmentation optimization method according to claim 4, characterized in that: The step of determining the fixed template corresponding to each section of the work order text according to the matching result includes: For each section of the work order text, if the text keyword set of the section of the work order text matches a consistent template keyword set, or matches a template keyword set with a similarity within a preset range, or matches a template keyword set with the same meaning, then the template containing the template keyword set is used as the fixed template for the section of the work order text.

6. The word segmentation optimization method according to claim 5, characterized in that: The step of determining the fixed template corresponding to each section of the work order text according to the matching result also includes: For each section of the work order text, if the work order text matches multiple template keyword sets, the keyword overlap between the text keyword set of the work order text and each matched template keyword set is calculated, and the template containing the template keyword set with the highest keyword overlap is used as the fixed template for this section of the work order text.

7. The word segmentation optimization method according to claim 1, characterized in that: The step of extracting target text data from each section of the work order text according to the template content and template rules corresponding to each fixed template includes: For each section of the work order text, if the template rule corresponding to the fixed template of the section of the work order text is to delete the template and the content after it, then all the text content of the section of the work order text is deleted; if the template rule corresponding to the fixed template of the section of the work order text is not to delete the content after the template, then regular matching is used to delete the text content in the section of the work order text that corresponds to the template content of the fixed template to obtain the target text data of the section of the work order text.

8. A word segmentation optimization device, characterized in that: The device comprises: A customer service work order acquisition module is used to acquire an initial customer service work order and pre-process the initial customer service work order to obtain a target customer service work order; A customer service work order segmentation module is used to segment the target customer service work order to obtain multiple work order texts; A fixed template retrieval module, used to retrieve the fixed template corresponding to each section of the work order text in the template library; The target text data extraction module is used to extract the target text data from each section of the work order text according to the template content and template rules corresponding to each fixed template, so as to perform word segmentation processing on each target text data.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the word segmentation optimization method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the word segmentation optimization method according to any one of claims 1 to 7 are performed.

Images