Automatic order dispatching method of platform and related equipment

Abstract

The embodiment of the invention provides an automatic order sending method of a platform and related equipment, and belongs to the technical field of computers. The method comprises the following steps: acquiring work order information needing to be dispatched; inputting the work order information into a large language model based on cue word engineering to obtain a first-level prediction result; if the first-level prediction result contains a preset value, starting an auxiliary model, and inputting the work order information and the first-level prediction result into the auxiliary model to obtain a second-level prediction result; and fusing the first-level prediction result and the second-level prediction result to obtain an order sending result. According to the invention, order sending prediction is carried out through the large language model and the auxiliary model, so that the accuracy of order sending prediction is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an automatic order dispatching method and related equipment for a platform. Background Technology

[0002] With the advancement of digital transformation in related services, the demand for hotline systems for these businesses is becoming increasingly widespread. Traditional work order dispatch methods mainly rely on manual judgment, which is inefficient, error-prone, and unable to cope with the ever-increasing volume of business. In recent years, with the development of artificial intelligence, especially natural language processing technology, there has been an attempt to use AI models to assist in work order classification and dispatch.

[0003] In related technologies, pre-trained models such as BERT are used to perform semantic analysis on work order texts and automatically match the most suitable department for processing. However, most of these systems rely on a single model for judgment, resulting in lengthy input texts and poor prediction accuracy.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose an efficient platform for automatic order dispatching and related equipment.

[0006] To achieve the above objectives, one aspect of this application proposes an automatic order dispatching method for a platform. The method includes: acquiring work order information requiring dispatch; inputting the work order information into a large language model based on prompt word engineering to obtain a first-level prediction result; if the first-level prediction result contains a preset value, activating a sub-model, inputting the work order information and the first-level prediction result into the sub-model to obtain a second-level prediction result; and fusing the first-level prediction result and the second-level prediction result to obtain an order dispatching result. This application uses a large language model and a sub-model for order dispatching prediction, which helps improve the accuracy of order dispatching prediction.

[0007] In some embodiments, the method further includes:

[0008] The work order information is processed by keyword matching using a rule engine;

[0009] If a match is successful, the matching result will be used as the order dispatch result.

[0010] If no match is found, return to the step of inputting the work order information into a large language model based on prompt word engineering to obtain the first-level prediction result.

[0011] In some embodiments, inputting the work order information into a large language model based on prompt word engineering to obtain a first-level prediction result includes:

[0012] The work order information is input into a large language model based on the first prompt word to obtain the first prediction result;

[0013] Based on the first prediction result, a second prompt word is determined, and the work order information is input into a large language model based on the second prompt word to obtain a second prediction result;

[0014] Based on the second prediction result, the first-level prediction result is determined.

[0015] In some embodiments, inputting the work order information and the primary prediction result into the sub-model to obtain the secondary prediction result includes:

[0016] If the first-level prediction result is a department that is not at the preset level, the output second-level prediction result will be a functional department.

[0017] Alternatively, if the first-level prediction result is a preset-level department, the address field in the work order information is encoded, and the output second-level prediction result is determined to be a preset-level department based on the encoded address field.

[0018] In some embodiments, the method further includes:

[0019] The large language model and the sub-model are iterated through the model hot-loading interface to obtain the updated large language model and the sub-model.

[0020] In some embodiments, fusing the primary prediction result and the secondary prediction result to obtain the order dispatch result includes:

[0021] If the reliability of the dispatch result is lower than the preset reliability threshold, it will be transferred to manual processing;

[0022] Alternatively, if the department corresponding to the dispatch result is not standardized, the dispatch result shall be updated according to the actual functions of the department;

[0023] Alternatively, if the work order information includes preset problem information, the department in the dispatch result will be updated to the next higher level department.

[0024] In some embodiments, the method further includes:

[0025] The dispatch results are encapsulated to obtain structured information;

[0026] The structured information is then returned to the order dispatcher.

[0027] To achieve the above objectives, another aspect of this application provides an automatic order dispatching device for a platform, the device comprising:

[0028] The acquisition module is used to obtain work order information that needs to be dispatched;

[0029] The first-level prediction module is used to input the work order information into a large language model based on prompt word engineering to obtain the first-level prediction result;

[0030] The secondary prediction module is used to start the sub-model if the primary prediction result contains a preset value, and input the work order information and the primary prediction result into the sub-model to obtain the secondary prediction result;

[0031] The order dispatch module is used to merge the first-level prediction results and the second-level prediction results to obtain the order dispatch results.

[0032] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0033] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0034] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0035] The embodiments of this application include at least the following beneficial effects: This application provides an automatic order dispatching method, apparatus, electronic device, storage medium, and program product for a platform. The method of this application includes: obtaining work order information that needs to be dispatched; inputting the work order information into a large language model based on prompt word engineering to obtain a first-level prediction result; if the first-level prediction result contains a preset value, activating a sub-model, inputting the work order information and the first-level prediction result into the sub-model to obtain a second-level prediction result; and fusing the first-level prediction result and the second-level prediction result to obtain the order dispatching result. This application uses a large language model and a sub-model for order dispatching prediction, which helps to improve the accuracy of order dispatching prediction. Attached Figure Description

[0036] Figure 1 This is a flowchart of the automatic order dispatching method of the platform provided in the embodiments of this application;

[0037] Figure 2 This is a flowchart of another automatic order dispatching method provided in the embodiments of this application;

[0038] Figure 3 This is a schematic diagram of the structure of the automatic order dispatching device of the platform provided in the embodiments of this application;

[0039] Figure 4This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0041] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0042] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0044] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0045] Large Language Model (LLM) refers to a natural language processing model based on deep learning. Compared with the AI ​​models used in the past, it has a large number of parameters and strong understanding and generation capabilities.

[0046] Bidirectional Encoder Representations from Transformers (BERT) is a pre-trained natural language processing model.

[0047] Text classification AI models: A type of AI model whose main function is to automatically determine the category of a text input. Typically, the input is a long text, and the output is the most likely option from a fixed set of choices. Common applications include: sentiment analysis (e.g., determining whether a comment is positive, negative, or neutral), news classification, spam detection, and intent recognition.

[0048] An Application Programming Interface (API) is used for communication between different software modules.

[0049] JavaScript Object Notation (JSON) is a lightweight data interchange format.

[0050] Flask: A lightweight Python web framework commonly used to build RESTful API services.

[0051] Artificial intelligence (AI) refers to the technology that enables machines to simulate human intelligent behavior.

[0052] Natural Language Processing (NLP) is a branch of artificial intelligence that studies how to understand and generate human language.

[0053] Prompt engineering is a technique that uses specific language instructions to guide a large model to output the desired content in the expected way, without requiring fine-tuning of the large model during training.

[0054] Structured: Organized in a fixed format (such as JSON) to facilitate subsequent system calls and processing.

[0055] Unstructured: Information with irregular or incomplete structure, without a predefined data model, and inconvenient to represent using a two-dimensional logical table in a database. This includes all formats of office documents, text, images, various reports, images, and audio / video information.

[0056] With the advancement of digital transformation in related services, the demand for hotline systems for related businesses is becoming increasingly widespread. Traditional work order dispatch methods mainly rely on manual judgment, which is inefficient, error-prone, and unable to cope with the ever-increasing volume of business.

[0057] In recent years, with the development of artificial intelligence, especially natural language processing (NLP) technology, attempts have begun to use AI models to assist in work order classification and dispatch. For example, pre-trained models such as BERT are used to perform semantic analysis on work order texts and automatically match the most appropriate handling department. However, most of these systems rely on a single model for judgment, lack the constraints of rule mechanisms, and the input text is often lengthy, resulting in limited accuracy, especially when dealing with multi-level responsibility divisions and complex policy interpretations.

[0058] Among related technologies, a more advanced approach is to combine a rule engine with a text classification AI model for work order dispatch. For example:

[0059] One implementation uses a BERT-based text classification model to perform semantic analysis on work order content, and combines a few hard rules (such as keyword matching) to correct the model output. The actual effect of the system is: 60% of work orders can be dispatched through the model, 40% cannot be dispatched, and among the 60% of dispatched work orders, there is a 93% accuracy rate.

[0060] In summary, the main drawbacks of existing technologies are analyzed as follows:

[0061] Uncontrollable model decision-making: Existing systems often rely on a single type of model for decision-making, lacking a rule-based fallback mechanism, which makes them prone to misjudgment;

[0062] Rules and models are separated: rule judgment and model reasoning are often independent processes, without forming a unified logical chain, resulting in information redundancy or conflict;

[0063] Loose deployment structure: Most systems do not clearly separate the main model from the sub-models, resulting in complex call chains and high response latency;

[0064] Lack of maintainability: The rules update and model iteration lack standardized processes, resulting in poor system scalability and maintainability in different working system environments;

[0065] High system iteration cost: The base model on which such a dispatch system relies usually needs to be retrained every 3-4 months to adapt to new rules / department name changes. Training a new model is often time-consuming, and the effectiveness of each training is unknown, requiring repeated testing.

[0066] Input length limitation: Due to architectural constraints, the models used in this type of system limit the length of the input text. However, in actual applications, the reports or complaints received by the system often contain lengthy and detailed descriptions. If the text length exceeds the model's limit, the system will forcibly truncate the input content. This results in the system receiving a cropped input, which affects the judgment result.

[0067] In view of this, this application provides an automatic work order dispatching scheme for a platform. This scheme proposes an intelligent work order dispatching system that integrates prompt word engineering and a multi-stage text classification model. By introducing the following core technical means, this scheme significantly improves the system's accuracy, robustness, and maintainability:

[0068] 1. Construct a large language model reasoning mechanism based on prompt word engineering.

[0069] In existing technologies, Large Language Models (LLMs) typically use raw text as input, lacking effective guidance from semantic context, leading to uncontrollable output results. This invention designs a structured prompt template to convert user-input work order content into structured information, and combines this with a pre-defined rule dictionary to limit options, enabling the large model to make better decisions based on the structured information and the limited options.

[0070] 2. The collaborative reasoning mechanism between the primary and secondary models improves classification accuracy.

[0071] In existing technologies, most systems use a single-type model for judgment, which cannot handle multi-level responsibility division in complex scenarios. This invention proposes a primary and secondary model collaborative reasoning mechanism:

[0072] The main model (LLM) uses the mechanism mentioned in section 1 as the main model, which is responsible for the judgment of the first-level department and achieves high semantic understanding ability based on prompt word engineering.

[0073] Sub-models (such as BERT): These are activated when the main model returns specific values ​​(such as "departments to be determined", "towns and streets"). They employ multi-level traditional text classification models to further refine the assignment of units to street offices or specific departments.

[0074] Comprehensive judgment module: integrates the results of rules, main model and sub-model to determine the final dispatching department.

[0075] This hierarchical reasoning mechanism not only improves the overall accuracy of the system, but also enhances its ability to handle rare edge cases.

[0076] 3. Deep integration of rule engine and model reasoning enhances system controllability.

[0077] In traditional approaches, rule-based judgment and model inference are often two independent processes, leading to logical conflicts and redundant processing. This invention achieves deep integration of rules and models through a combination of pre-rule implementation, model fallback, and post-processing correction.

[0078] Rule-based matching: For work orders that clearly fall within the scope of the rules (such as operator, water and electricity transfer, etc.), the rule engine makes the determination directly, skipping model reasoning.

[0079] Model fallback judgment: Work orders that do not match the rules are handed over to the main and auxiliary model systems for reasoning.

[0080] Post-processing correction mechanism: Corrects business logic such as address validity verification, illegal construction reassignment, and property management classification of the model output.

[0081] This fusion mechanism ensures the certainty of rules in key scenarios while retaining the generalization ability of large models to judge unknown situations, making the judgment process more flexible.

[0082] 4. Supports dynamic rule updates and hot model loading, improving system maintainability.

[0083] Existing systems often require retraining a new AI model when rules change or models are iterated, impacting departmental dispatch efficiency. This invention provides a dynamic rule loading mechanism and a model hot-loading interface, ensuring that the system can complete configuration updates without service interruption, greatly improving system availability and maintainability.

[0084] The automatic order dispatch method of the platform provided in this application relates to the field of computer technology. The automatic order dispatch method of the platform provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, etc., but is not limited thereto; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application implementing the platform's automatic order dispatch method, etc., but is not limited to the above forms.

[0085] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0086] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.

[0087] Figure 1 This is an optional flowchart of the automatic order dispatching method of the platform provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S100 to S400.

[0088] Step S100: Obtain the work order information that needs to be dispatched;

[0089] Step S200: Input the work order information into a large language model based on prompt word engineering to obtain the first-level prediction result;

[0090] Step S300: If the first-level prediction result contains a preset value, start the sub-model, input the work order information and the first-level prediction result into the sub-model to obtain the second-level prediction result;

[0091] Step S400: Combine the first-level prediction results and the second-level prediction results to obtain the order dispatch result.

[0092] In this application, the work order information can be sent by the requester through a terminal. After receiving the work order information, this application preprocesses it and then inputs the preprocessed work order information into a large language model to obtain a primary prediction result. The primary prediction result is a preliminary order dispatch result, and the preset values ​​can be preset address information, name information, etc. The secondary model refines the primary prediction result. If the primary prediction result does not contain preset values, the primary prediction result is verified for compliance, and then the order dispatch result is obtained based on the primary prediction result.

[0093] In some embodiments, the method further includes:

[0094] The work order information is processed by keyword matching using a rule engine;

[0095] If a match is successful, the matching result will be used as the order dispatch result.

[0096] If no match is found, return to the step of inputting the work order information into a large language model based on prompt word engineering to obtain the first-level prediction result.

[0097] In some embodiments, inputting the work order information into a large language model based on prompt word engineering to obtain a first-level prediction result includes:

[0098] The work order information is input into a large language model based on the first prompt word to obtain the first prediction result;

[0099] Based on the first prediction result, a second prompt word is determined, and the work order information is input into a large language model based on the second prompt word to obtain a second prediction result;

[0100] Based on the second prediction result, the first-level prediction result is determined.

[0101] This application provides a prompt word update strategy, which updates the prompt words based on the prediction results of the large language model and then makes predictions again through the large language model. Of course, this application does not limit the number of iterations of the prompt words.

[0102] In some embodiments, inputting the work order information and the primary prediction result into the sub-model to obtain the secondary prediction result includes:

[0103] If the first-level prediction result is a department that is not at the preset level, the output second-level prediction result will be a functional department.

[0104] Alternatively, if the first-level prediction result is a preset-level department, the address field in the work order information is encoded, and the output second-level prediction result is determined to be a preset-level department based on the encoded address field.

[0105] This application determines the department level of the secondary forecast results based on the department level of the primary forecast results.

[0106] In some embodiments, the method further includes:

[0107] The large language model and the sub-model are iterated through the model hot-loading interface to obtain the updated large language model and the sub-model.

[0108] In some embodiments, fusing the primary prediction result and the secondary prediction result to obtain the order dispatch result includes:

[0109] If the reliability of the dispatch result is lower than the preset reliability threshold, it will be transferred to manual processing;

[0110] Alternatively, if the department corresponding to the dispatch result is not standardized, the dispatch result shall be updated according to the actual functions of the department;

[0111] Alternatively, if the work order information includes preset problem information, the department in the dispatch result will be updated to the next higher level department.

[0112] The preset confidence threshold is the acceptable prediction accuracy. The preset problem information includes complaint information, conflict information, etc. If the work order information includes the preset problem information, the department in the dispatch result is replaced with the higher-level department to obtain the updated dispatch result.

[0113] In some embodiments, the method further includes:

[0114] The dispatch results are encapsulated to obtain structured information;

[0115] The structured information is then returned to the order dispatcher.

[0116] The method provided in this application will be described in detail below with specific application examples:

[0117] The core objective of this invention is:

[0118] This paper proposes a master model reasoning mechanism based on prompt word engineering to improve the accuracy and interpretability of large language models in government work order dispatching tasks.

[0119] Construct a primary and secondary model collaborative reasoning architecture to achieve refined order dispatching under multi-level responsibility division.

[0120] A mechanism for deep integration of rules and models is implemented, balancing automation efficiency with the need for manual control.

[0121] It provides dynamic rule updates and hot model loading capabilities, enhancing the system's scalability and ease of maintenance.

[0122] Through the above-mentioned technological innovations, this invention can significantly improve the intelligence level of work order dispatch, reduce the cost of manual intervention, and improve the response efficiency of related work, and has broad promotion value and application prospects.

[0123] Reference Figure 2 As shown, the dispatching method provided in this application includes the following steps:

[0124] Step S21: Client requests input: Original work order information sent by the government affairs platform system.

[0125] Step S22, Work Order Preprocessing Module: Cleans, standardizes, and completes fields for the original work order content.

[0126] Step S23, Rule Matching Judgment: Quickly determine whether specific conditions are met based on preset rules.

[0127] Step S24, Main Model Inference Module: Based on the Large Language Model (LLM) of the prompt word engineering, it is used for first-level judgment.

[0128] Step S25, Sub-model Inference Module: Activated when the main model outputs a special value, further refining the dispatch unit.

[0129] Step S26, Comprehensive Judgment and Correction Module: Integrates post-processing rules and main and secondary model results to determine the final dispatch unit and make business logic corrections.

[0130] Step S27: Generate final dispatch result: Encapsulate complete dispatch information, including whether a dispatch has been made, the handling department, confidence level, etc.

[0131] Step S28: Return the client response: Return structured information to the service caller.

[0132] about Figure 2 The system modules are described in detail.

[0133] ① Work order preprocessing module.

[0134] Function Description:

[0135] Clean the original work order data submitted by users.

[0136] Fill in the missing fields.

[0137] Standardize the processing of special characters such as newline characters and spaces.

[0138] Build a unified text field for use in subsequent models.

[0139] ② Rule matching module.

[0140] Function Description:

[0141] Quickly identify whether it belongs to certain fixed rule scenarios (such as water and electricity transfer, illegal construction, etc.);

[0142] If the rule is matched, the corresponding order dispatch result is returned directly, skipping the model reasoning stage.

[0143] Technical Implementation: A keyword matching mechanism is used. Multiple preset key fields are scanned. If a match is found, the corresponding dispatching department and rule type are returned.

[0144] ③Main model reasoning module.

[0145] Function Description:

[0146] Receive the pre-processed work order content.

[0147] Build a structured prompt word template.

[0148] Call LLM for semantic understanding and departmental judgment.

[0149] Output first-level order dispatch suggestions.

[0150] Technical Implementation: Load the LLM. Build a prompt word template, limiting the list of selectable departments. Retrieve results by calling the public LLM API or a local LLM deployment service.

[0151] Detailed explanation of the role of prompt word engineering:

[0152] In this invention, prompt word engineering serves as a bridge connecting user intent and model output, and its role is mainly reflected in the following aspects:

[0153] Standardized output format: By pre-setting field descriptions and structured format requirements, we ensure that the model output meets business needs. For example:

[0154] The prompt word template is constructed from the following JSON-based dictionary object or a list of dictionary objects:

[0155] The "{item}" field contains non-fixed content. It's created by cleaning the text to remove special symbols and blanks when the user inputs information, before being added to the prompt template to elicit an AI model's response. Adjusting the content, structure, and expression of the prompt template guides the AI ​​model to output the desired result; revising the prompt template is one method of prompt engineering.

[0156] Supports multi-turn interaction mechanisms: New prompts are constructed based on the model's output to guide the model in gradually refining field information. For example:

[0157] When the AI ​​model receives the first question indicating that the handling department is the "Market Supervision Administration", it will further construct prompts to ask a second round of questions, inquiring whether the current user input information involves any consumer-related reports or complaints. If the AI ​​model answers "yes", the handling department will be changed to "Consumer Council"; otherwise, the original handling department will be maintained.

[0158] ④ Sub-model inference module

[0159] Function Description:

[0160] This is triggered when the main model outputs special values ​​such as "manual processing by the system platform" or "offices of various towns and streets".

[0161] Traditional text classification models are used to make judgments based on address and event category.

[0162] Output the specific street or department name.

[0163] Technical Implementation: Load a text classification model. Input fields from the user request (excluding address information, such as request title, location, and complaint details) after previous text cleaning are fed into the model. If the result indicates a department other than a subdistrict office, the model outputs the department with the highest probability. If the previous level's judgment infers a subdistrict office, the address field is encoded, and the address information is used to determine the subdistrict office. The model outputs the subdistrict office with the highest probability and greater than the expected probability threshold; otherwise, it is forwarded to the system platform for manual processing.

[0164] ⑤ Comprehensive Judgment and Correction Module

[0165] Function Description:

[0166] The results of merging rules, main model, and sub-model.

[0167] Determine whether to dispatch the order or send it back for manual review.

[0168] The model output is then reassigned to the actual dispatching unit.

[0169] The record needs to be manually reviewed.

[0170] Technical Implementation:

[0171] Set a confidence threshold for the judgment result (e.g., if the confidence of the result obtained by the sub-model system is lower than a certain threshold, it will not be assigned to manual processing). Determine the legality of the address and rules based on the rule table and the original work order information. Determine whether to transfer the operation to another unit based on the department name in the reasoning result. For example, when the reasoning result is "XX Street Office," the result will be modified to "XX Street Comprehensive Administrative Law Enforcement Team" based on the actual functions of that street office; when the result is "XX Street Comprehensive Administrative Law Enforcement Team" and the original input information contains words related to wage arrears, the handling department will be modified to the street office that is the superior of that law enforcement team.

[0172] ⑥ Additional notes for the sub-model module

[0173] The sub-model system is a crucial component of this invention, assisting the main model in work order dispatching. Its core function is to further analyze the "location of the incident" information in the work order when the main model cannot directly determine the specific handling department (e.g., the main model returns "local department"). This analysis identifies the specific street or town to which the address belongs, and based on this, makes the final dispatching decision. One difference between the model used in this system and the large model used in the main system is that this sub-model needs to acquire and clean data, and then use this data for training to achieve the best matching results.

[0174] Training process overview:

[0175] Training data source: Historical work order data collected from government hotline platforms (such as the "12345" citizen service hotline), covering real dispatch records for different time periods (e.g., from Month X, 20XX to Month X, 20YY). The data comes from multiple Excel files, containing fields such as "Location of Incident" and "Secondary Department Handling," and is used for training and validation of the address model.

[0176] Data split: 85% for training set and 15% for validation set.

[0177] Model Structure: The address classification model implements text classification based on the BERT architecture and employs adversarial training (FGM) to improve model robustness. Two models are used: the first model determines whether the order belongs to the local street office, and the second model further determines the appropriate department to which the order should be sent.

[0178] Training parameter settings:

[0179] Learning rate: 8e-6;

[0180] Batch Size: 8 during training, 4 during validation;

[0181] Maximum sequence length: 512;

[0182] Loss function: FocalLoss, which alleviates the class imbalance problem;

[0183] Adversarial training: FGM, to enhance the model's generalization ability;

[0184] Optimizer: AdamW;

[0185] Early Stop: If F1 does not improve after 3 consecutive rounds of verification, training is stopped.

[0186] Model evaluation metrics:

[0187] Continuously monitor Accuracy and Micro-F1 on the validation set;

[0188] At the end of each epoch, the current best Acc is output, and the model weights are saved;

[0189] The final model achieves an accuracy of over 92% on the validation set;

[0190] The core of this invention lies in constructing a structured extraction system and method for market supervision information based on a large language model. Through prompt word engineering and a rule knowledge base, it achieves the function of automatically dispatching complaint or report work orders extracted from the 12345 system to street offices or non-street office departments. Specifically,

[0191] ① A Large Language Model (LLM) inference mechanism based on cue word engineering.

[0192] Using structured prompt templates, the original work order content is formatted and input into a large-scale model for inference. (The key is the content of these prompt templates; the content and number of templates will continuously increase based on actual needs. For example, the first prompt template is usually used to initially obtain the scope of a handling department or directly obtain the name of a specific department; the second prompt template, based on the department obtained from the previous results, may need to further derive the actual name of the assigning department, depending on its specificity; and so on, with more templates gradually guiding the AI ​​model to generate results. The information required in each template comes from the user's input.) Based on the inference results and a rule mapping dictionary, options are limited, enabling the large language model to make accurate decisions within a limited scope. It can be extended to other government service scenarios and has good transferability.

[0193] ② Collaborative reasoning mechanism between primary and secondary models.

[0194] A hierarchical reasoning architecture of "main model + sub-model" is proposed. The main model is used for judgment by primary departments (such as the market supervision bureau). The sub-model is activated when the main model returns a specific value (such as "manual processing" or "township / street offices"), and is used to further refine the decision to the street or specific department. It achieves refined task dispatching under multi-level responsibility division (the key point lies in the specific design of the main model mobilizing the sub-model). Hot loading of the model is supported, facilitating later updates and maintenance.

[0195] ③ Deep integration mechanism of rules and deep learning models

[0196] Rule Pre-processing: Work orders clearly falling within the rule scope (such as operator registration, utility transfers, etc.) are directly determined by the rule engine. Model Backup: Work orders that do not match the rules are handled by the main and auxiliary models for reasoning. Post-processing Correction: The model output is corrected for business logic such as address validity verification, illegal construction reassignment, and property management classification. This solves the problem of disconnect between rules and models, achieves a unified logical chain, and improves the overall accuracy and controllability of the system (the key lies in the interaction between this rule table and the AI ​​model).

[0197] ④ Street-level address classification model based on BERT model (sub-model).

[0198] This document describes a training method for a BERT text classification model specifically designed for identifying departments / local subdistrict offices. When the input field is content other than the incident location, the output is a non-subdistrict-level department name; otherwise, if the input field is the incident location, the output is a subdistrict-level attribution judgment (the key point lies in the workflow logic of the sub-model system). When the main model returns ambiguous results such as "location" or "subdistrict," the sub-model is triggered to perform precise matching. It supports dynamically updating the department and corresponding number mapping list to adapt to changes in department names.

[0199] ⑤ Dynamic rule update mechanism.

[0200] All rules are implemented as functions, supporting add, modify, and delete operations. Rules are executed before models to ensure determinism in critical scenarios. External file configuration of rules is supported. Uninterrupted updates can be achieved by reloading the configuration file.

[0201] ⑥ Interpretability mechanisms under multi-model and rule-based hybrid decision-making.

[0202] Each work order is labeled with the source of the judgment (rule, AI model recognition, rule after AI model recognition). Output fields include interpretable information such as "rule type," "judgment logic," and "confidence level." This allows agencies and operations personnel to quickly locate the cause of errors. It complies with the auditing requirements and traceability standards of government systems.

[0203] In summary, this invention is a closed-loop management mechanism for intelligent government affairs that utilizes large language models and prompt word engineering. From receiving data, preprocessing, rule judgment, model reasoning, comprehensive judgment to result return, the entire process is configurable, monitorable, and auditable, and supports closed-loop processes such as manual review, model feedback, and error correction. Apart from the key points mentioned in ①-④, the entire mechanism's operational logic itself is the object of protection.

[0204] Compared with existing technologies, the master-slave model system of this invention has significant advantages in several aspects. Currently, the Huangpu Star system relies on traditional text classification models, requiring frequent fine-tuning and periodic model updates to adapt to new work order types and business changes, resulting in high maintenance costs. In contrast, this system employs a large language model inference mechanism based on prompt word engineering, combined with a rule engine and deep learning models, achieving rapid adaptation to new business scenarios without frequent training. Furthermore, through the master-slave model collaborative inference mechanism, the system achieves highly accurate street-level address recognition and departmental judgment, enabling 93% of work orders to be automatically dispatched by the model with an accuracy rate of 96%, far exceeding the existing system's 60% automatic dispatch rate and 93% accuracy rate.

[0205] From a deployment and operation perspective, this system supports hot model reloading and seamless service upgrades, exhibiting excellent scalability and stability. The deep integration mechanism between rules and models ensures consistency and controllability in dispatching under complex business logic, while log tracking and anomaly recording mechanisms facilitate subsequent optimization and analysis. Furthermore, the system supports multi-version model coexistence, structured output format log storage, and high-concurrency access, providing solid technical support for the efficient operation of government hotlines. These advantages not only enhance the intelligence level of government services but also lay a solid foundation for the continuous evolution of future government AI systems.

[0206] Please see Figure 3 This application also provides an automatic order dispatching device for a platform, which can implement the above-described method. The device includes:

[0207] The acquisition module 610 is used to acquire work order information that needs to be dispatched;

[0208] The first-level prediction module 620 is used to input the work order information into a large language model based on prompt word engineering to obtain the first-level prediction result;

[0209] The secondary prediction module 630 is used to start the sub-model if the primary prediction result contains a preset value, input the work order information and the primary prediction result into the sub-model, and obtain the secondary prediction result.

[0210] The order dispatch module 640 is used to merge the first-level prediction results and the second-level prediction results to obtain the order dispatch results.

[0211] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0212] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0213] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0214] Please see Figure 4 , Figure 4 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0215] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0216] The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application.

[0217] The input / output interface 903 is used to implement information input and output;

[0218] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0219] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0220] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0221] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0222] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0223] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0224] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0225] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0226] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0227] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0228] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0229] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0230] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0231] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0232] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0233] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0234] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0235] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0236] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An automatic order dispatching method for a platform, characterized in that, The method includes the following steps: Obtain the work order information that needs to be dispatched; The work order information is input into a large language model based on prompt word engineering to obtain the first-level prediction result; If the primary prediction result contains a preset value, the secondary model is activated, and the work order information and the primary prediction result are input into the secondary model to obtain the secondary prediction result; The order dispatch result is obtained by combining the first-level prediction result and the second-level prediction result.

2. The method according to claim 1, characterized in that, The method further includes: The work order information is processed by keyword matching using a rule engine; If a match is successful, the matching result will be used as the order dispatch result. If no match is found, return to the step of inputting the work order information into a large language model based on prompt word engineering to obtain the first-level prediction result.

3. The method according to claim 1, characterized in that, The step of inputting the work order information into a large language model based on prompt word engineering to obtain a first-level prediction result includes: The work order information is input into a large language model based on the first prompt word to obtain the first prediction result; Based on the first prediction result, a second prompt word is determined, and the work order information is input into a large language model based on the second prompt word to obtain a second prediction result; Based on the second prediction result, the first-level prediction result is determined.

4. The method according to claim 1, characterized in that, The step of inputting the work order information and the first-level prediction result into the sub-model to obtain the second-level prediction result includes: If the first-level prediction result is a department that is not at the preset level, the output second-level prediction result will be a functional department. Alternatively, if the first-level prediction result is a preset-level department, the address field in the work order information is encoded, and the output second-level prediction result is determined to be a preset-level department based on the encoded address field.

5. The method according to claim 1, characterized in that, The method further includes: The large language model and the sub-model are iterated through the model hot-loading interface to obtain the updated large language model and the sub-model.

6. The method according to claim 1, characterized in that, The order dispatch result is obtained by fusing the primary forecast result and the secondary forecast result, including: If the reliability of the dispatch result is lower than the preset reliability threshold, it will be transferred to manual processing; Alternatively, if the department corresponding to the dispatch result is not standardized, the dispatch result shall be updated according to the actual functions of the department; Alternatively, if the work order information includes preset problem information, the department in the dispatch result will be updated to the next higher level department.

7. The method according to claim 1, characterized in that, The method further includes: The dispatch results are encapsulated to obtain structured information; The structured information is then returned to the order dispatcher.

8. An automatic order dispatching device for a platform, characterized in that, The device includes: The acquisition module is used to obtain work order information that needs to be dispatched; The first-level prediction module is used to input the work order information into a large language model based on prompt word engineering to obtain the first-level prediction result; The secondary prediction module is used to start the sub-model if the primary prediction result contains a preset value, and input the work order information and the primary prediction result into the sub-model to obtain the secondary prediction result; The order dispatch module is used to merge the first-level prediction results and the second-level prediction results to obtain the order dispatch results.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images