Hotline work order classification method and system based on large model

Abstract

The invention relates to the technical field of large model application, in particular to a hotline work order classification method and system based on a large model, and the method comprises the following steps: hotline work order data analysis and arrangement: carrying out the cleaning, screening and refining of the hotline work order data, so as to obtain input data meeting the work order classification task requirements; work order classification task management: performing overall planning on task management, task related data management and task related prompt management; executing a work order classification task: completing the work order classification task according to the residual margin relationship between the target classification list and the token when the large model API is called; the method has the beneficial effects that based on the general language processing capacity of a large model, the hot line work orders are classified in combination with a prompt prompt word project, the data analysis and mining capacity of the hot line work orders is improved, important event problems are focused, and the hot problems reflected by the concern of the masses are better solved.

Description

Technical Field

[0001] The present invention relates to the technical field of large model applications, and in particular to a hotline work order classification method and system based on a large model. Background Art

[0002] In hotline scenarios, a large number of hotline tickets are generated daily. When accepting a ticket, agents categorize it, including by ticket type, the category of the matter to which it belongs, the emotional state of the caller, and whether it is an emergency ticket. This classification greatly aids in the analysis and mining of tickets, and is an essential, general, and fundamental classification. In reality, the dimensions of hotline ticket classification can be more flexible and diverse, especially when analyzing hotline tickets for specialized topics, where the classification dimensions are often more specific. Relying solely on general, basic classifications is insufficient for analysis and mining. With the application of big models and AI in the hotline sector, expectations for the application value that hotline ticket-based data can provide have increased. Leveraging the general language processing capabilities of big models, tackling the more complex problem of hotline ticket classification has become a natural choice.

[0003] The patent "Method and System for Hot Word Mining, Classification, and Analysis Based on Work Orders" completes the work order classification task by performing word frequency statistics on a large amount of work order data and obtaining keywords. These keywords are then used to create corresponding topic categories. However, in reality, classification categories are often given, and categories often condense more abstract matters that may not be expressed by the keywords in the work order content. Summary of the Invention

[0004] The purpose of the present invention is to provide a hotline work order classification method and system based on a large model to solve the problems raised in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a hotline work order classification method based on a large model, comprising the following steps:

[0006] Hotline work order data analysis and collation: Clean, filter, and refine hotline work order data to obtain input data that meets the requirements of work order classification tasks;

[0007] Work order classification task management: Overall planning of task management, task-related data management, and task-related prompt management;

[0008] Work order classification task execution: Complete the work order classification task based on the relationship between the target classification list and the remaining token when the large model API is called.

[0009] Preferably, the hotline work order data analysis and collation process specifically includes:

[0010] Data cleaning and screening: Remove work order data with empty key data and sort out unreasonable formats in key data; select relevant data columns based on classification goals to obtain input data that can participate in the work order classification task;

[0011] Work order data extraction: Considering the token quantity limit of the large model, the cleaned and filtered work order data is refined to reduce its content size to a limited range. Refinement methods include extracting keyword phrases or extracting brief descriptions of the work order data. The refined limited range must be maintained during prompt data maintenance.

[0012] Preferably, the work order classification task management process specifically includes:

[0013] Task management: Maintain the classification of task objectives and use a database to maintain this data;

[0014] Task-related data management: Maintain the data after cleaning and screening in the P1 phase and the data with classification labels generated after task execution. Due to the relatively large amount of data, object storage is used for maintenance;

[0015] Task-related prompt management: Maintain the prompts used in work order classification tasks and use a database to maintain this data.

[0016] Preferably, the work order classification task execution process specifically includes:

[0017] Calculate the remaining token balance for large model API calls: The remaining token balance for large model API calls is equal to the token limit for large model API calls minus the number of characters in the prompt template, minus the limit for work order data extraction, that is, W_target category = W_total - W_template - W_data;

[0018] Processing is performed based on the relationship between the target classification list content and the remaining token balance.

[0019] Preferably, during the execution of the work order classification task, the work order classification strategy is dynamically adjusted based on the comparison result of the target classification list size and the remaining balance of the token when the large model API is called, specifically including: when the content of the target classification list is smaller than the remaining balance of the token, the model API is directly called and the classification is completed according to a specific prompt; when the content of the target classification list is larger than the remaining balance of the token, the target classification list is split and the model API is called multiple times, and the model is called again according to the results to ensure that the hotline work order is accurately classified while meeting the token limit of the large model API call.

[0020] A system for hotline work order classification based on a large model, comprising:

[0021] Hotline work order data analysis and collation module, used to clean, filter and refine hotline work order data to obtain input data that meets the requirements of work order classification tasks;

[0022] The work order classification task management module is used to coordinate and plan task management, task-related data management, and task-related prompt management;

[0023] The work order classification task execution module is used to complete the work order classification task based on the relationship between the target classification list and the remaining token when the large model API is called.

[0024] Preferably, the hotline work order data analysis and collating module further includes:

[0025] The data cleaning and screening unit is used to remove work order data with empty key data and sort out unreasonable formats in key data; according to the classification target, relevant data columns are selected to obtain input data that can participate in the work order classification task;

[0026] The work order data extraction unit is used to extract the cleaned and filtered work order data, reducing its content size to a limited range, taking into account the token number limit of the large model. The extraction methods include extracting keyword phrases or extracting brief descriptions of the work order data, and the extraction limit range must be maintained during prompt data maintenance.

[0027] Preferably, the work order classification task management module further includes:

[0028] The task management unit is used to maintain the classification of task objectives and use a database to maintain the data;

[0029] The task-related data management unit is used to maintain the cleaned and filtered data in the hotline work order data analysis and collation module, as well as the data with classification labels generated after task execution. Due to the relatively large amount of data, object storage is used for maintenance;

[0030] The task-related prompt management unit is used to maintain the prompts used in the work order classification tasks, and the database is used to maintain the data.

[0031] Preferably, the work order classification task execution module further includes:

[0032] The token quantity calculation unit is used to calculate the remaining token balance when calling the large model API. The remaining balance is equal to the token limit when calling the large model API minus the number of characters in the prompt template, minus the limit on the work order data extraction, that is, W_target category = W_total - W_template - W_data;

[0033] The classification strategy adjustment unit is used to dynamically adjust the work order classification strategy based on the relationship between the target classification list content and the remaining token balance.

[0034] Preferably, when dynamically adjusting the work order classification strategy, the classification strategy adjustment unit further includes:

[0035] Direct classification unit: When the target classification list is less than the remaining token, it directly calls the model API and completes the classification according to the specific prompt.

[0036] Split the classification unit. When the content of the target classification list is larger than the remaining token, split the target classification list and call the model API multiple times. Then call the model again based on the results to ensure that the hotline work order is accurately classified while meeting the token limit of the large model API call.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The hotline work order classification method and system based on the big model proposed in the present invention realizes the classification of hotline work orders based on the general language processing capability of the big model and combines the prompt word engineering, improves the data analysis and mining capabilities of hotline work orders, focuses on important event issues, and better solves the hot issues that the public is concerned about and reflects; to realize the classification of hotline work orders, the general language capability of the big model is the basis for the implementation of the method, and the prompt word engineering is a flexible method for solving classification tasks, avoiding the previous heavy customization of models for text classification tasks, eliminating unnecessary model training processes, and improving the versatility of the solution to the work order classification task. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Flow chart of the method of the present invention;

[0040] Figure 2 This is a flowchart of the work order task management entity of the present invention;

[0041] Figure 3 This is a flowchart of the work order classification task execution of the present invention. DETAILED DESCRIPTION

[0042] In order to clearly and completely describe the objectives and technical solutions of the present invention and make the advantages more clearly understood, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present invention, not all of them, and are only used to explain the embodiments of the present invention, not to limit the embodiments of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] For example 1, please refer to Figures 1 to 3 The present invention provides a technical solution: a hotline work order classification method based on a large model, comprising the following steps:

[0044] P1: Work order classification task management

[0045] Work order classification task management includes task management, task-related data management, and task-related prompt management.

[0046] In task management, it mainly includes the maintenance of task target classification categories. This data is usually not large and the data volume is limited, so it can be maintained by a database. Task-related data mainly includes the maintenance of data after cleaning and screening in the P1 stage, and the maintenance of data generated by classification labels after task execution. This data volume is relatively large and can be maintained by object storage, such as minio. The management of task-related prompts is mainly the maintenance of prompts used in work order classification tasks. This data volume is not large and can be maintained by a database. Related entity diagrams are as follows: Figure 2 shown.

[0047] P2: Hotline work order data analysis and collation

[0048] Hotline work order data is the basis of classification tasks. In actual work order classification tasks, the hotline work order data provided is often multi-dimensional and heterogeneous, so analysis and sorting work is essential.

[0049] S1: Data cleaning and screening

[0050] Typical data cleaning tasks include data cleaning and data filtering. Data cleaning typically involves removing work order data with empty key data and sorting out any inappropriate formatting within the key data. Data filtering involves selecting relevant data columns based on the classification objective. After cleaning and filtering, data becomes the input data for the work order classification task.

[0051] S2: Work order data extraction

[0052] When executing work order classification tasks, it's important to consider the token limit of large models. The input data for invoking large models in work order classification tasks primarily consists of three parts: a prompt template, work order data, and a list of work order classification targets. The prompt template is typically of a fixed size, while the work order data, derived from real-world business scenarios, has a variable size. Therefore, it's necessary to refine the work order data to reduce its size to within a specified range.

[0053] Since the limited scope of work order data extraction is related to the token limit when calling the large model API, this type of data needs to be maintained when maintaining prompt data in the P1 stage.

[0054] Work order data extraction can be divided into multiple dimensions, ranging from extracting key words to extracting brief descriptions of the work order data. Different extraction methods result in different information losses, which should be considered during data extraction and subsequent prompt word design. Work order data extraction can also be achieved by combining a large model with prompt words. For example, the following prompt words can be used:

[0055] You are a hotline intelligence information analyst proficient in natural language processing, responsible for improving the efficiency and quality of hotline services. You will use advanced natural language processing technology to deeply analyze and summarize call texts, accurately identify the content of citizens' conversations, ignore the content of agents, and generate work order summaries to summarize the details of the received issues.

[0056] Output format: where and what happened, the subject, subject, location involved, and the user's demands or requests.

[0057] constraint:

[0058] 1. Be sure to extract according to the content. Do not make up names of people, places, or times, and do not add irrelevant content information.

[0059] 2. Carefully read the transcript, distinguishing between the citizen's and the agent's statements. Avoid including the agent's own. Extract key information from the citizen's statement: the event, location, and appeal. Describe the event in as much detail as possible, ensuring coherence and correcting any typos.

[0060] 3. Do not include too many subjects in the summary. The total output word count should not exceed {} words

[0061] Please follow the prompts above to ensure that each work order summary meets the requirements and accurately reflects the demands of citizens.

[0062] The following is a sample output, including the ticket content and a standard answer. Please refer to the standard answer format for output.

[0063] Here is an example:

[0064] Example 1: xxx

[0065] Answer 1: xxx

[0066] End of Example

[0067] Please summarize the work order according to {} and output it in the format of the standard answer

[0068] In the above prompt, the input data are the number of words that need to be limited and the work order data content.

[0069] P3: Work order classification task execution

[0070] During the work order classification process, the target classification list also needs to consider the token limit for large model API calls. The target classification list is related to the current work order classification task and is flexible. When classifying work orders, it is necessary to compare the target classification list size with the remaining token balance for large model API calls to determine how to proceed. The remaining token balance for large model API calls is equal to the token limit for large model API calls minus the number of characters in the prompt template, minus the limit on work order data extraction, as shown in the following formula:

[0071] W 目标分类 =W 总 -W 模板 -W 数据

[0072] S1: If the target category list content is less than the token remaining

[0073] You can complete the work order classification directly through the following prompt

[0074] ###Character Scene

[0075] You are a hotline intelligence information analyst proficient in natural language processing, responsible for improving the efficiency and quality of hotline services. You need to use advanced natural language processing technology to classify the provided ticket content into a target category within the provided topic.

[0076] The current theme is {}

[0077] The target categories related to the current topic are

[0078] {}

[0079] Output content constraints

[0080] 1. Please first determine whether the work order content belongs to the current topic. If it does not belong to the topic, no category judgment will be performed.

[0081] 2. If the ticket content belongs to the current topic, determine if the ticket content belongs to one of the provided categories and return only one of them. If none of them match, return None.

[0082] 3. The output format is required to be returned in json format, as shown below

[0083] {{"Topic":"...","Category":"..."}}

[0084] ###Provide an example

[0085] Example 1: xxx

[0086] Answer 1: xxx

[0087] End of Example

[0088] =================

[0089] The work order content is as follows:

[0090] {}

[0091] S2: If the target category list content is greater than the remaining token

[0092] You can split the target category list into multiple parts, ensuring that each part is smaller than the remaining tokens, and then call the model API multiple times. In this case, a result item will be generated each time. If each result belongs to the same category, you need to call the model API again to determine the result. This process can be implemented according to the divide-and-conquer design principle.

[0093] The work order classification task execution process is as follows Figure 3 shown.

[0094] In the second embodiment, based on the first embodiment, a system for hotline work order classification based on a large model is proposed, including:

[0095] The hotline work order data analysis and collation module is used to clean, filter, and refine the hotline work order data to obtain input data that meets the requirements of the work order classification task; it further includes:

[0096] The data cleaning and screening unit is used to remove work order data with empty key data and to organize unreasonable formats in the key data; according to the classification target, relevant data columns are selected to obtain input data that can participate in the work order classification task; the work order data refining unit is used to refine the cleaned and filtered work order data, taking into account the token number limit of the large model, and reduce its content size to within a limited range; the refining methods include refining keyword phrases or refining brief descriptions of the work order data, and the refined limited range needs to be maintained when maintaining prompt data.

[0097] The work order classification task management module is used to coordinate the management of tasks, task-related data, and task-related prompts; it further includes:

[0098] The task management unit is used to maintain the classification categories of task objectives, and the database is used to maintain this data; the task-related data management unit is used to maintain the data that has been cleaned and screened in the hotline work order data analysis and collation module, as well as the data with classification labels generated after task execution. Due to the relatively large amount of data, object storage is used for maintenance; the task-related prompt management unit is used to maintain the prompts used in the work order classification tasks, and the database is used to maintain this data.

[0099] The work order classification task execution module is used to complete the work order classification task based on the relationship between the target classification list and the remaining balance of the token when the large model API is called; it further includes: a token quantity calculation unit, which is used to calculate the remaining balance of the token when the large model API is called. The remaining balance is equal to the token limit when the large model API is called minus the number of characters in the prompt template, and then minus the limited number of work order data extraction, that is, W_target classification = W_total-W_template-W_data; a classification strategy adjustment unit, which is used to dynamically adjust the work order classification strategy according to the relationship between the target classification list content and the token remaining balance; it further includes: a direct classification unit, when the target classification list content is less than the token remaining balance, the model API is directly called and the classification is completed according to a specific prompt; a split classification unit, when the target classification list content is greater than the token remaining balance, the target classification list is split and the model API is called multiple times, and the model judgment is called again according to the results to ensure that the hotline work order is accurately classified while meeting the token limit of the large model API call.

[0100] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A hotline work order classification method based on a large model, characterized by: The following steps are involved: Hotline work order data analysis and collation: Clean, filter, and refine hotline work order data to obtain input data that meets the requirements of work order classification tasks; Work order classification task management: Overall planning of task management, task-related data management, and task-related prompt management; Work order classification task execution: Complete the work order classification task based on the relationship between the target classification list and the remaining token when the large model API is called.

2. The hotline work order classification method based on a large model according to claim 1 is characterized by: The hotline work order data analysis and collation process specifically includes: Data cleaning and screening: Remove work order data with empty key data and sort out unreasonable formats in key data; select relevant data columns based on classification goals to obtain input data that can participate in the work order classification task; Work order data extraction: Considering the token quantity limit of the large model, the cleaned and filtered work order data is refined to reduce its content size to a limited range. Refinement methods include extracting keyword phrases or extracting brief descriptions of the work order data. The refined limited range must be maintained during prompt data maintenance.

3. The hotline work order classification method based on a large model according to claim 2 is characterized by: The work order classification task management process specifically includes: Task management: Maintain the classification of task objectives and use a database to maintain this data; Task-related data management: Maintain the data after cleaning and screening in the P1 phase and the data with classification labels generated after task execution. Due to the relatively large amount of data, object storage is used for maintenance; Task-related prompt management: Maintain the prompts used in work order classification tasks and use a database to maintain this data.

4. The hotline work order classification method based on a large model according to claim 3 is characterized by: The work order classification task execution process specifically includes: Calculate the remaining token balance for large model API calls: The remaining token balance for large model API calls is equal to the token limit for large model API calls minus the number of characters in the prompt template, minus the limit for work order data extraction, that is, W_target category = W_total - W_template - W_data; Processing is performed based on the relationship between the target classification list content and the remaining token balance.

5. The hotline work order classification method based on a large model according to claim 4 is characterized in that: During the execution of the work order classification task, the work order classification strategy is dynamically adjusted based on the comparison result between the target classification list size and the remaining token when the large model API is called. Specifically, when the content of the target classification list is less than the remaining token, the model API is directly called and the classification is completed according to the specific prompt; when the content of the target classification list is larger than the remaining token, the target classification list is split and the model API is called multiple times, and the model is called again based on the results to ensure that the hotline work order is accurately classified while meeting the token limit of the large model API call.

6. A system for the hotline work order classification method based on a large model according to claim 5, characterized in that: include: Hotline work order data analysis and collation module, used to clean, filter and refine hotline work order data to obtain input data that meets the requirements of work order classification tasks; The work order classification task management module is used to coordinate and plan task management, task-related data management, and task-related prompt management; The work order classification task execution module is used to complete the work order classification task based on the relationship between the target classification list and the remaining token when the large model API is called.

7. A system according to claim 6, characterized in that: The hotline work order data analysis and collation module further includes: The data cleaning and screening unit is used to remove work order data with empty key data and sort out unreasonable formats in key data; according to the classification target, relevant data columns are selected to obtain input data that can participate in the work order classification task; The work order data extraction unit is used to extract the cleaned and filtered work order data, reducing its content size to a limited range, taking into account the token number limit of the large model. The extraction methods include extracting keyword phrases or extracting brief descriptions of the work order data, and the extraction limit range must be maintained during prompt data maintenance.

8. A system according to claim 7, characterized in that: The work order classification task management module further includes: The task management unit is used to maintain the classification of task objectives and use a database to maintain the data; The task-related data management unit is used to maintain the cleaned and filtered data in the hotline work order data analysis and collation module, as well as the data with classification labels generated after task execution. Due to the relatively large amount of data, object storage is used for maintenance; The task-related prompt management unit is used to maintain the prompts used in the work order classification tasks, and the database is used to maintain the data.

9. A system according to claim 8, characterized in that: The work order classification task execution module further includes: The token quantity calculation unit is used to calculate the remaining token balance when calling the large model API. The remaining balance is equal to the token limit when calling the large model API minus the number of characters in the prompt template, minus the limit on the work order data extraction, that is, W_target category = W_total - W_template - W_data; The classification strategy adjustment unit is used to dynamically adjust the work order classification strategy based on the relationship between the target classification list content and the remaining token balance.

10. A system according to claim 9, characterized in that: When dynamically adjusting the work order classification strategy, the classification strategy adjustment unit further includes: Direct classification unit: When the target classification list is less than the remaining token, it directly calls the model API and completes the classification according to the specific prompt. Split the classification unit. When the content of the target classification list is larger than the remaining token, split the target classification list and call the model API multiple times. Then call the model again based on the results to ensure that the hotline work order is accurately classified while meeting the token limit of the large model API call.

Images