Multi-modal processing work order intelligent customer service management method and system

Abstract

This invention discloses a method and system for intelligent customer service management of work orders based on multimodal processing. The method includes: deriving and preprocessing work order data from input data; extracting features from the work order data using a multimodal encoder; constructing a multimodal contrastive loss function using contrastive learning to map feature data from different modalities to a unified semantic embedding space; establishing cross-modal semantic contrast through a cross-attention mechanism, which can be fused into a multimodal semantic feature vector; inputting the multimodal semantic feature vector into a large language inference model for inference, and interactively reasoning within the context window of the large language inference model from a dynamic knowledge graph to obtain the structured understanding result of the current work order; and automatically dispatching work orders by matching dispatchers based on the structured understanding result. This invention solves the problems of existing work order processing methods, such as the inability to fuse cross-modal data and poor real-time performance of work order dispatch.

Description

Technical Field

[0001] This invention relates to the field of intelligent customer service management, and in particular to a work order intelligent customer service management method and system based on multimodal processing. Background Technology

[0002] Work order management is widely used in IT operations, telecommunications, government hotlines, e-commerce after-sales service, and other fields. With the deepening of enterprise digital transformation, work order data is becoming increasingly diverse and heterogeneous. User-submitted work order data is no longer limited to text; it increasingly appears in the form of audio, images, or structured forms. However, traditional work order processing solutions mainly rely on manual classification and dispatch or simple rule-based automation, facing significant challenges in terms of accuracy, processing capacity, and information utilization.

[0003] In recent years, although there has been an increasing number of applications of artificial intelligence technology in the field of work order processing, it often has the following limitations: (1) Existing work order processing solutions often only process single modal data, or simply splice together different modal data after separate processing, lacking cross-modal feature alignment and semantic fusion mechanisms; (2) Most existing work order processing solutions are mainly based on static knowledge bases or rule bases to classify and dispatch work orders, and cannot dynamically supplement new knowledge as work orders are processed, resulting in problems such as poor data real-time performance, static models, and low recommendation accuracy in work order dispatch.

[0004] Based on the aforementioned shortcomings, there is an urgent need for an intelligent customer service management solution for work orders that can fully integrate multimodal data and realize dynamic knowledge evolution. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, one of the objectives of this invention is to provide a work order intelligent customer service management method based on multimodal processing, which can solve the problems of existing work order processing being unable to integrate across modalities and poor real-time work order dispatch.

[0006] The second objective of this invention is to provide a work order intelligent customer service management system based on multimodal processing, which can solve the problems of existing work order processing being unable to integrate across modalities and poor real-time work order dispatch.

[0007] One of the objectives of this invention is achieved through the following technical solution: A work order intelligent customer service management method based on multimodal processing, the work order intelligent customer service management method includes: Data acquisition steps: Acquire the input data of the current work order and derive the work order data based on the input data of the work order data, and perform corresponding preprocessing on the data of different modalities in the work order data; Feature extraction steps: The preprocessed work order data is extracted using a multimodal encoder built by the system to obtain feature data of different modalities. A multimodal contrastive loss function is constructed by combining contrastive learning to map the feature data of different modalities to a unified semantic embedding space. At the same time, a cross-modal semantic contrast is established through a cross-attention mechanism to fuse the feature data of different modalities into a multimodal semantic feature vector. Model reasoning steps: The multimodal semantic feature vector of the current work order is input into the pre-trained big language reasoning model of the system for reasoning. During the reasoning process, multiple historical work orders similar to the current work order are obtained from the dynamic knowledge graph constructed by the system and input into the context window of the big language reasoning model for interactive reasoning, thereby obtaining the structured understanding result of the current work order. Work order dispatching steps: Based on the structured understanding results of the current work order and the customer service personnel in the system, the corresponding dispatcher is determined and the current work order is pushed to the corresponding dispatcher.

[0008] Furthermore, the step of pushing the current work order to the corresponding dispatcher in the work order dispatching process specifically includes: automatically storing the current work order in the work queue of the corresponding dispatcher, and pushing the work order summary information of the current work order to the corresponding dispatcher; and estimating the response waiting time of the corresponding dispatcher in real time, and when the response waiting time of the corresponding dispatcher exceeds the preset response time, matching the next customer service handler from the customer service handlers in the system and using them as the dispatcher to push the current work order to the corresponding dispatcher.

[0009] Furthermore, the work order dispatching step, which matches the structured understanding result of the current work order with the customer service personnel in the system to determine the corresponding dispatcher, specifically includes: dynamically maintaining the multi-dimensional capability tags of each customer service personnel in real time based on the processing status and processing data of each work order in the system; calculating the matching score of each tag of each customer service personnel based on the structured understanding result of the current work order, and then obtaining the comprehensive score of each customer service personnel based on the matching score of each tag; sorting each customer service personnel based on their comprehensive scores and determining the optimal customer service personnel based on the sorting result, and using them as the dispatcher; the multi-dimensional capability tags of each customer service personnel include skill tags, historical processing success rate, average response time, average processing time, current workload, customer satisfaction rating, online status, and scheduling information.

[0010] Furthermore, the intelligent customer service management method for work orders also includes a solution recommendation step: matching the corresponding standard processing flow template from the dynamic knowledge graph based on the classification result of the current work order, and combining the big language reasoning model to personalize the corresponding standard processing flow template according to the specific characteristics of the current work order to generate processing suggestion steps for the current work order; Simultaneously, based on the current work, relevant historical work order solutions are retrieved from the dynamic knowledge graph, and multiple historical work orders are sorted according to semantic similarity and effect score. Several candidate historical work orders are then selected based on the sorting results. Furthermore, a draft solution for the current work order is generated based on the solutions of the several candidate historical work orders, and the draft solution is pushed to the corresponding dispatcher.

[0011] Furthermore, the intelligent customer service management method for work orders also includes: an adaptive learning step: after the corresponding work order is processed in the system, feedback information for the corresponding work order is generated by collecting the processing result data of the corresponding work order, and the feedback information of the corresponding work order is input into the dynamic knowledge graph to update the dynamic knowledge graph, and the complete processing process of the corresponding work order is added to the dynamic knowledge graph as a new case node; the updating of the dynamic knowledge graph specifically includes: updating the success rate and average time consumption of existing case nodes, adjusting the weight of skill tags according to the work order processing behavior data of the corresponding customer service personnel, and establishing new inter-entity relationships; the feedback information of the corresponding work order includes: processing result feedback information, actual processing time consumption, solution effectiveness score, dispatch accuracy rate, customer satisfaction, and behavioral data during the processing.

[0012] Furthermore, the data acquisition step specifically includes the following preprocessing steps for work order data based on data type: performing spelling correction and standardization on the original text data; performing speech recognition on the original speech data using an industry-specific speech recognition model to convert it into corresponding text data; and performing image preprocessing on the original image data based on a visual encoder; while acquiring the work order data for the current work order, a globally unique identifier is generated for the current work order.

[0013] Furthermore, the feature extraction step specifically includes: Step 1: The large language reasoning model, pre-trained based on the general language model, performs text encoding on the text data to extract deep semantic features, and performs visual extraction on the original image data based on the visual deep learning model or CLIP visual encoder to obtain visual features, thereby obtaining the text modality features and visual modality features of the current work. Step 2: Construct a multimodal contrastive loss function based on the contrastive learning strategy, and construct positive sample pairs and negative sample pairs based on the features of different modal data of the same work order. By minimizing the contrastive loss, feature vectors of different modalities of the same work order are made closer to each other in the embedding space, while semantic vectors of different work orders are made further apart. Step 3: Construct cross-modal semantic alignment through a cross-attention mechanism. Use text modality features as the query matrix and image modality features as the key and value matrices. Calculate cross-modal attention output to establish the correspondence between text descriptions and image regions, thereby achieving accurate matching between text and images. Then, fuse the text modality features and visual modality features of the current work order to obtain multimodal semantic features.

[0014] Further, the model reasoning steps specifically include: Step 1: Generating a query vector for the current work order based on the work order data of the current work order, and calculating the similarity between the query vector of the current work order and each historical work order in the dynamic knowledge graph within the system to obtain multiple candidate historical work orders and construct a first candidate set; Querying the dynamic knowledge graph based on the entity and classification label identified by the current work order to obtain work order nodes directly connected to the entity of the current work order, and querying work order nodes indirectly associated with the entity of the current work order, and then obtaining corresponding candidate historical work orders based on the work order nodes and constructing a second candidate set; Merging and deduplicating the historical work orders in the first candidate set and the historical work orders in the second candidate set, calculating the comprehensive similarity of each candidate historical work order, and then obtaining multiple historical work orders similar to the current work order based on the comprehensive similarity and recording them as similar historical work orders; Step 2: Based on each similar historical work order and the dynamic knowledge graph, derive the solution, associated information graph profile, and process node knowledge for each similar historical work order, and form them into structured prompt words. These, along with the multimodal semantic feature vector of the current work order, are input into the context window of the large language reasoning model. This allows the large language reasoning model to generate the intent recognition and classification, key entity recognition and knowledge graph entity linking, and emergency procedure and impact scope determination results for the current work order based on the structured prompt words provided by the dynamic knowledge graph, thereby deriving the structured understanding result. The structured understanding result includes classification labels, priority scores, a list of associated knowledge graph entities, clusters of similar historical work orders, and possible root cause hypotheses inferred by the large language reasoning model.

[0015] Furthermore, in the model reasoning step, when the large-scale model reasoning model identifies new entities, new attributes, or newly inferred relationships during the reasoning process, the current work order and the new entities, new attributes, or newly inferred relationships are fed back into the dynamic knowledge graph in real time to construct new entity nodes, new attribute data, and new relationships within the dynamic knowledge graph. Specifically, when the large-scale language reasoning model identifies entities not stored in the dynamic knowledge graph, candidate entity nodes are generated based on the entities, and the relationships between related entities are inferred based on the context to calculate the confidence score of the relationships. When the confidence score is greater than a preset threshold, the new entity is automatically written into the dynamic knowledge graph.

[0016] The second objective of this invention is achieved by the following technical solution: A multimodal processing-based intelligent customer service management system for work orders, comprising: The data acquisition module is used to acquire the input data of the current work order and derive the work order data based on the input data of the work order data, and to perform corresponding preprocessing on the data of different modalities in the work order data. The feature extraction module is used to extract features from the preprocessed work order data through the multimodal encoder built by the system to obtain feature data of different modalities. It also combines contrastive learning to construct a multimodal contrastive loss function to map the feature data of different modalities to a unified semantic embedding space. At the same time, it establishes cross-modal semantic contrast through a cross-attention mechanism to fuse the feature data of different modalities into a multimodal semantic feature vector. The model reasoning module is used to input the multimodal semantic feature vector of the current work order into the pre-trained big language reasoning model of the system for reasoning. During the reasoning process, multiple historical work orders similar to the current work order are obtained from the dynamic knowledge graph constructed by the system and input into the context window of the big language reasoning model for interactive reasoning, thereby obtaining the structured understanding result of the current work order. The work order dispatch module is used to match the structured understanding result of the current work order with the customer service personnel in the system to determine the corresponding dispatcher and push the current work order to the corresponding dispatcher.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention extracts features from different modalities in work order data and fuses them to associate the data between different modalities. Then, it combines a large language reasoning model and a dynamic knowledge graph to perform reasoning, thereby deriving the reasoning result of the current work order. Finally, it matches the reasoning result of the current work order with customer service personnel in the system to select the optimal customer service personnel for automatic work order dispatch. This solves the problems of existing work orders being unable to achieve cross-modal fusion and poor real-time work order dispatch. Attached Figure Description

[0018] Figure 1 The flowchart of the intelligent customer service management method for work orders based on multimodal processing provided by the present invention is shown. Detailed Implementation

[0019] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments. Example 1

[0020] To address the shortcomings of existing work order processing methods, this invention provides a work order intelligent customer service management method based on multimodal processing. This method combines multimodal work order input data to achieve semantic fusion of multimodal data, and simultaneously coordinates work order classification and dispatch to achieve dynamic semantic updates, significantly improving work order processing efficiency and the real-time nature of work order dispatch. Figure 1 As shown, the present invention provides a preferred embodiment of a work order intelligent customer service management method based on multimodal processing, comprising: Step S1: Obtain the input data of the current work order and derive the work order data based on the input data of the current work order, and perform corresponding preprocessing on the data of different modalities in the work order data.

[0021] Preferably, the work order data in this invention includes data of multiple modalities, such as text data, image data, and voice data. Different technical means are used to perform corresponding preprocessing based on the different modalities of the work order data. Specifically, text data undergoes spell correction and standardization; for voice data, a speech recognition module or model combining industry content is used to convert audio data into text data, while also recognizing prosodic features in the voice data; for image data, image preprocessing is performed on the original image data based on a visual encoder. For example, let's take a work order received in a broadband fault handling scenario from a telecommunications operator, specifically regarding slow broadband speeds, as an example: In actual use, when users find that the broadband speed is slow, they can call the hotline to report it: slow broadband speed, games are particularly laggy after 8 pm, or enter the corresponding text through the webpage or official account: slow broadband speed, games are particularly laggy at night, frequent disconnections, etc. They can also upload a photo of the optical modem indicator light status and a screenshot of the game lag through the official account or webpage.

[0022] The system can generate the following work order data based on the user's input: Text data: Slow broadband speed, severe lag when playing games at night, frequent disconnections. This can be addressed by performing corresponding pinyin correction or standardized writing on the text data.

[0023] Image data: Several screenshots of terminal error messages, each displaying the error code XXXXXX. Visual encoders are used to preprocess the screenshots to identify the text: Error Code XXXXXX.

[0024] Audio data: The audio content is: slow broadband speed, and games lag significantly after 8 PM. It is converted into text data using an ASR model incorporating an IT terminology database, and prosodic features of the audio signal, such as tone of voice and pace, are extracted.

[0025] At the same time, when the system receives the current work order, it also sets a globally unique identifier for the current work order so that the work order can be uniquely queried and matched based on the globally unique identifier.

[0026] Step S2: The preprocessed work order data is extracted by the multimodal encoder constructed by the system to obtain feature data of different modalities. The multimodal contrastive loss function is constructed by contrastive learning to map the feature data of different modalities to a unified semantic embedding space. At the same time, cross-modal semantic contrast is established through cross-attention mechanism to fuse the feature data of different modalities into a multimodal semantic feature vector.

[0027] For text data, pre-trained large language inference models are used to encode text data, audio content converted to text data, or directly acquired text data to extract deep semantic features. For example, large language inference models such as Qwen2.5 and DeepSeek are used, and a general model is trained based on historical data from the corresponding industry before being applied to text encoding to extract key deep semantic features, such as slow broadband speed, after 8 PM, game lag, and game disconnection. Simultaneously, when text data is converted to audio data, prosodic features are extracted, such as whether the user's speaking speed is too fast or the volume is too high, to identify emotional characteristics, such as impatience.

[0028] For image data, visual features can be extracted using a visual Transformer or CLIP visual encoder. The LOS indicator is flashing red, indicating that the optical signal is lost; the PON indicator is solid green, indicating that the optical network terminal is registered normally; game screen lag and stuttering.

[0029] More preferably, a contrastive learning strategy is employed to construct a multimodal contrastive loss function, mapping feature data from different modalities to a unified semantic embedding space to achieve global data consistency. Specifically, positive samples are constructed from different modalities of the same work order, and negative samples are constructed from modalities of different work orders. By minimizing the contrastive loss, the semantic vectors of different modalities within the same work order are brought closer together in the embedding space, while the semantic vectors of different work orders are moved further apart. Simultaneously, text modal feature data is used as the query matrix, and image modal feature data is used as the key and value matrices. Cross-modal attention output is calculated to establish a correspondence between text data and image data, achieving precise matching between elements in the text and elements in the image data. The multimodal data is then fused to derive a multimodal semantic vector.

[0030] Specifically, through a cross-attention mechanism, using text features as queries and image features as keys and values, a cross-modal correspondence is established between service connectivity failures and error codes to rule out server connectivity issues caused by network disconnections. By associating and fusing feature data from different modalities, a multimodal semantic feature vector is generated. For example, through the aforementioned cross-modal alignment, the time period feature (after 8 AM) can be cross-modally associated with the optical signal fault status (LOS light flashing red) to form a comprehensive judgment of optical signal anomalies during peak hours.

[0031] Step S3: Input the multimodal semantic feature vector of the current work order into the pre-trained big language reasoning model of the system for reasoning. During the reasoning process, multiple historical work orders similar to the current work order are obtained from the dynamic knowledge graph constructed by the system and input into the context window of the big language reasoning model for interactive reasoning, so as to obtain the structured understanding result of the current work order.

[0032] Specifically, the large-scale language inference model is pre-trained based on existing general-purpose models in the market combined with historical data related to industry content. Examples include mainstream international closed-source models such as OpenAI's GPT series, Anthropic's Claude series, and Google's Gemini series, as well as mainstream domestic models like DeepSeek, Qwen, and Doubao. By deploying locally and performing customized fine-tuning, it achieves semantic inference for specific industries, covering various deployment scenarios. When using the model for inference, multiple large models can be combined; for example, DeepSeek can be used to handle high-concurrency classification tasks, Claude for inference logic, and Qwen for private deployment to handle sensitive customer data.

[0033] Meanwhile, while employing a large language reasoning model, this invention also injects historical work order data into the large language reasoning model through a dynamically constructed knowledge graph to assist in reasoning. This not only improves the accuracy of the reasoning results of the large language reasoning model, but also feeds back the identified new data to the dynamic knowledge graph to achieve dynamic updates of the knowledge within the dynamic knowledge graph, ensuring timely data updates.

[0034] More preferably, the dynamic knowledge graph is pre-built based on historical data. Specifically, it is constructed by acquiring a large amount of historical work orders, business knowledge, organizational structure, customer master data and external knowledge sources for data mining. At the same time, the core node types and edge relationships of the knowledge graph are defined based on the intelligent work order management scenario, and then the construction of the dynamic knowledge graph is realized. For example, historical work order data includes work order ID, title, description text, attachments, processing records, solutions, processing personnel, timestamps, customer information, satisfaction ratings, etc. This data can be used to construct work order nodes, problem type nodes, solution nodes, personnel nodes, and customer nodes. Similarly, data in the business knowledge base, including product manuals, fault code tables, SOP documents, FAQs, and service catalogs, can be used to predefine problem classification systems, standard solution templates, equipment models, and specifications. Organizational structure data includes customer service team personnel information, skill tags, schedules, and permission roles, which can be used to construct customer service personnel nodes and capability profiles. Customer master data includes customer ID, name, level, region, and contract information, which can be used to construct customer nodes and their attributes. External knowledge sources include equipment manufacturer knowledge bases, industry standard terminology, and publicly available technical documents, used to supplement domain terminology and entity definitions. By cleaning, segmenting, and deduplicating the above text data, and extracting multimodal features from image or audio data, structured tags are obtained.

[0035] Specifically, the multimodal semantic feature vectors generated by fusion are input into a pre-trained large language reasoning model for reasoning. During the reasoning process, multiple historical work orders similar to the current work order are retrieved from the dynamic knowledge graph constructed by the system. Structured prompts are generated based on the solutions, associated information graphs, and process node knowledge of each historical work order, and then injected into the context window of the large language reasoning model. The number of historical work orders similar to the current work order retrieved from the dynamic knowledge graph can be dynamically adjusted based on the work order complexity and similarity distribution; for example, selecting 5-10 is sufficient.

[0036] During the reasoning process, this invention injects historical work order data into a large language reasoning model to assist in reasoning. Guided by the structured prior knowledge provided by the knowledge graph, the large language reasoning model will deduce the following results: work order intent identification and type, key entity identification and knowledge graph entity linking, and urgency and impact scope judgment. Specifically, work order intent identification and type not only identifies the category of the work order, such as network failure, but also identifies more specific types, such as: Wi-Fi connection failure - incorrect password, or Wi-Fi connection failure - weak signal. Key entity identification and knowledge graph entity linking links the entities mentioned in the work order to corresponding nodes in the knowledge graph; entities include, for example, device model, software version, and location. Urgency and impact scope judgment is used to comprehensively consider semantic content, entity attributes, and historical models to give a priority score for the current work order, thereby determining the priority of the current work order.

[0037] Furthermore, this invention also feeds back the results of the large language reasoning model to the knowledge graph to achieve dynamic updates of the knowledge graph, thereby enabling real-time dynamic updates of the knowledge graph and linking the large language reasoning model with the knowledge graph for feedback updates. That is, when the large language reasoning model derives new entities, new attributes, and new relationships, it feeds these new entities, attributes, and relationships back to the knowledge graph, which then updates its data based on these new entities, attributes, and relationships. Specifically, when the large language reasoning model identifies entities or concepts not existing in the knowledge graph, it can generate candidate entity nodes, infer new relationships between entities based on context, and calculate the confidence score of the new relationship. When the confidence score exceeds a certain threshold, the new relationship is considered reliable and is directly written into the knowledge graph. When the confidence score is less than a certain threshold, the new relationship is considered potentially uncertain, triggering a manual review process for manual evaluation.

[0038] Preferably, the large language reasoning model leverages its reasoning capabilities to infer multi-hop relationships based on existing relationships in the knowledge graph, thereby discovering potential hidden connections. Through collaborative reasoning between the large language reasoning model and the knowledge graph, it can output a structured understanding structure, specifically including: category labels, priority scores, a list of associated knowledge graph entities, historically similar work order clusters, and possible root cause hypotheses from the large language reasoning model. Category labels include not only the work order category but also the specific subcategory of the work order.

[0039] Specifically, after retrieving similar historical work orders from the dynamic knowledge graph, the following data can be obtained: The user's location is: XX Community, the network topology is GPON network, and the splitter model is 1:30; historical data shows that the community has experienced multiple issues of increased optical attenuation due to overload of the splitter port during the evening peak hours, and three similar work orders were found.

[0040] By injecting the above search results into the context window of the large language inference model in the form of structured prompts, it can be inferred that: the current fault is: insufficient optical power during GPON peak hours, not a problem with the user's indoor line, the work order priority is: high, and it is recommended to switch to a second-line network for priority processing.

[0041] Step S4: Based on the structured understanding results of the current work order and the customer service personnel in the system, the corresponding dispatcher is determined and the current work order is pushed to the corresponding dispatcher.

[0042] Specifically, the system dynamically maintains a competency profile for each customer service representative, including skill tags, historical success rate, average response time, average processing time, current workload, customer satisfaction rating, online status, and scheduling information. Skill tags refer to a representative's skills or areas of expertise, typically extracted from their basic information and historical work orders, along with their proficiency weights. Historical success rate refers to the success rate of a representative in handling past work orders, further subdivided by work order type. Average response time and average processing time are calculated based on historical work order response times and processing times. Current workload includes the number of pending work orders and estimated cumulative working hours. Customer satisfaction rating is the average customer evaluation score for historical work orders handled by the representative. Online status and scheduling information refer to the representative's current online status or scheduling information. Each representative's competency profile is dynamically updated in real-time based on the work order processing process within the system.

[0043] The system matches the structured output of the current work order with the competency profiles of each customer service representative within the system to select the appropriate representative for the current work order. Specifically, the system calculates a score for each customer service representative based on the current work order, which is calculated for each dimension of the competency profile and then weighted and merged according to each dimension.

[0044] For example, the skill matching score is calculated by determining the semantic similarity between the current work order's category tag and each customer service representative's skill tag.

[0045] Load balancing is calculated based on the number of pending work orders and estimated processing capacity for each customer service representative.

[0046] Historical performance matching can be derived based on the historical success rate of each customer service representative's work orders.

[0047] Geographic / time zone compatibility can be calculated based on the matching degree between the customer service representative's registered location and the location where the work order occurred.

[0048] Customer preference matching, for example, when a customer is a VIP or higher level, a senior service personnel can be matched for them.

[0049] The scores from the aforementioned multiple dimensions are weighted and fused to obtain a comprehensive score for each customer service representative. Then, based on this comprehensive score, the optimal representative is selected as the dispatcher, and the current work order is automatically assigned to them. Specifically, multiple customer service representatives are ranked from highest to lowest based on their comprehensive scores, and the representative with the highest comprehensive score is selected as the dispatcher.

[0050] More preferably, when the current work order is assigned to the dispatcher, the current work order is automatically stored in the corresponding dispatcher's work queue, and the work order summary information of the current work order is pushed to the corresponding dispatcher; and the response waiting time of the corresponding dispatcher is estimated in real time, and when the response waiting time of the corresponding dispatcher exceeds the preset response time, the next customer service handler is matched from the customer service handlers in the system and designated as the dispatcher to push the current work order to the corresponding dispatcher.

[0051] More preferably, when selecting the corresponding dispatcher, the present invention also generates processing suggestions based on the structured understanding of the current work order. That is, the present invention further includes: matching the corresponding standard processing flow template from a dynamic knowledge graph based on the classification tags of the current work order, and then using a large language reasoning model to personalize the standard processing flow template according to the specific characteristics of the current work order to generate processing suggestion steps for the current work order.

[0052] Meanwhile, based on the current work, relevant historical work orders solutions are retrieved from the dynamic knowledge graph, and multiple historical work orders are sorted according to semantic similarity and effect score. Several candidate historical work orders are then selected based on the sorting results. A draft solution for the current work order is generated based on the solutions of several candidate historical work orders, and the draft solution is pushed to the corresponding dispatcher.

[0053] For example, the system will automatically generate a work order for secondary network optimization based on the reasoning results, dispatch it to the West City Network Optimization Team, and assign a corresponding dispatcher to it. The dispatcher will also receive a note: "No need to replace the optical modem; please reassure the user and wait for network adjustments; estimated completion time is 3 hours." Once the dispatcher accepts the order, they can go to the corresponding area to perform the repair. After the repair is completed, the user's broadband will be restored to normal, and the work order will be confirmed as complete.

[0054] Meanwhile, during the actual handling process by customer service personnel, the system can continuously provide assistance through dialogue. For example, when customer service personnel encounter specific questions, they can ask questions to the larger model to obtain immediate answers.

[0055] More preferably, after the corresponding work order is processed within the system, feedback information for the corresponding work order is generated by collecting the processing result data and input into the dynamic knowledge graph to update the dynamic knowledge graph. The complete processing process of the corresponding work order is also added to the dynamic knowledge graph as a new case node. Specifically, updating the dynamic knowledge graph includes: updating the success rate and average processing time of existing case nodes, adjusting the weight of skill tags based on the work order processing behavior data of the corresponding customer service personnel, and establishing new inter-entity relationships. The feedback information for the corresponding work order includes: processing result feedback information, actual processing time, solution effectiveness score, dispatch accuracy rate, customer satisfaction, and behavioral data during the processing. Once the work order is completed, the customer service personnel confirm the problem is resolved and record their processing feedback result in the dynamic knowledge graph to establish an early warning rule node for subsequent proactive monitoring.

[0056] This invention enables the identification, classification, and automatic dispatch of work orders, and provides processing suggestions for each work order. Compared to traditional manual order dispatch, it significantly improves work order processing efficiency and shortens response time. Furthermore, during work order identification, this invention utilizes a large language reasoning model and a dynamic knowledge graph for collaborative reasoning, improving reasoning accuracy while also enabling dynamic updates to the knowledge graph within the system, ensuring timely updates to the knowledge base and enhancing the accuracy of work order dispatch. Example 2

[0057] Based on Embodiment 1, the present invention also provides another embodiment: a work order intelligent customer service management system based on multimodal processing, comprising: The data acquisition module is used to acquire the input data of the current work order and derive the work order data based on the input data of the work order data, and to perform corresponding preprocessing on the data of different modalities in the work order data. The feature extraction module is used to extract features from the preprocessed work order data through the multimodal encoder built by the system to obtain feature data of different modalities. It also combines contrastive learning to construct a multimodal contrastive loss function to map the feature data of different modalities to a unified semantic embedding space. At the same time, it establishes cross-modal semantic contrast through a cross-attention mechanism to fuse the feature data of different modalities into a multimodal semantic feature vector. The model reasoning module is used to input the multimodal semantic feature vector of the current work order into the pre-trained big language reasoning model of the system for reasoning. During the reasoning process, multiple historical work orders similar to the current work order are obtained from the dynamic knowledge graph constructed by the system and input into the context window of the big language reasoning model for interactive reasoning, thereby obtaining the structured understanding result of the current work order. The work order dispatch module is used to match the structured understanding results of the current work order with the customer service personnel in the system to determine the corresponding dispatcher and push the current work order to the corresponding dispatcher.

[0058] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A work order intelligent customer service management method based on multimodal processing, characterized in that, The intelligent customer service management method for work orders includes: Data acquisition steps: Acquire the input data of the current work order and derive the work order data based on the input data of the work order data, and perform corresponding preprocessing on the data of different modalities in the work order data; Feature extraction steps: The preprocessed work order data is extracted using a multimodal encoder built by the system to obtain feature data of different modalities. A multimodal contrastive loss function is constructed by combining contrastive learning to map the feature data of different modalities to a unified semantic embedding space. At the same time, a cross-modal semantic contrast is established through a cross-attention mechanism to fuse the feature data of different modalities into a multimodal semantic feature vector. Model reasoning steps: The multimodal semantic feature vector of the current work order is input into the pre-trained big language reasoning model of the system for reasoning. During the reasoning process, multiple historical work orders similar to the current work order are obtained from the dynamic knowledge graph constructed by the system and input into the context window of the big language reasoning model for interactive reasoning, thereby obtaining the structured understanding result of the current work order. Work order dispatching steps: Based on the structured understanding results of the current work order and the customer service personnel in the system, the corresponding dispatcher is determined and the current work order is pushed to the corresponding dispatcher.

2. The intelligent customer service management method for work orders based on multimodal processing according to claim 1, characterized in that, The work order dispatching step, specifically when pushing the current work order to the corresponding dispatcher, includes: automatically storing the current work order in the work queue of the corresponding dispatcher, and pushing the work order summary information of the current work order to the corresponding dispatcher; and estimating the response waiting time of the corresponding dispatcher in real time, and when the response waiting time of the corresponding dispatcher exceeds the preset response time, matching the next customer service handler from the customer service handlers in the system and using them as the dispatcher to push the current work order to the corresponding dispatcher.

3. The intelligent customer service management method for work orders based on multimodal processing according to claim 1, characterized in that, The work order dispatching step, which matches the structured understanding result of the current work order with the customer service personnel in the system to determine the corresponding dispatcher, specifically includes: dynamically maintaining multi-dimensional capability tags for each customer service personnel in real time based on the processing status and processing data of each work order in the system; calculating the matching score of each tag for each customer service personnel based on the structured understanding result of the current work order, and then obtaining the comprehensive score for each customer service personnel based on the matching score of each tag; sorting each customer service personnel according to their comprehensive scores and determining the optimal customer service personnel based on the sorting result, and using them as the dispatcher; the multi-dimensional capability tags for each customer service personnel include skill tags, historical processing success rate, average response time, average processing time, current workload, customer satisfaction rating, online status, and scheduling information.

4. The intelligent customer service management method for work orders based on multimodal processing according to claim 1, characterized in that, The intelligent customer service management method for work orders also includes a solution recommendation step: matching the corresponding standard processing flow template from the dynamic knowledge graph based on the classification result of the current work order, and combining the big language reasoning model to personalize the corresponding standard processing flow template according to the specific characteristics of the current work order to generate processing suggestion steps for the current work order. Simultaneously, based on the current work, relevant historical work order solutions are retrieved from the dynamic knowledge graph, and multiple historical work orders are sorted according to semantic similarity and effect score. Several candidate historical work orders are then selected based on the sorting results. Furthermore, a draft solution for the current work order is generated based on the solutions of the several candidate historical work orders, and the draft solution is pushed to the corresponding dispatcher.

5. The intelligent customer service management method for work orders based on multimodal processing according to claim 1, characterized in that, The intelligent customer service management method for work orders further includes: an adaptive learning step: after the corresponding work order is processed in the system, feedback information for the corresponding work order is generated by collecting the processing result data of the corresponding work order and inputting the feedback information of the corresponding work order into the dynamic knowledge graph to update the dynamic knowledge graph, and the complete processing process of the corresponding work order is added to the dynamic knowledge graph as a new case node; the updating of the dynamic knowledge graph specifically includes: updating the success rate and average time consumption of existing case nodes, adjusting the weight of skill tags according to the work order processing behavior data of the corresponding customer service personnel, and establishing new inter-entity relationships; the feedback information of the corresponding work order includes: processing result feedback information, actual processing time consumption, solution effectiveness score, dispatch accuracy rate, customer satisfaction, and behavioral data during the processing.

6. The intelligent customer service management method for work orders based on multimodal processing according to claim 1, characterized in that, The data acquisition step specifically includes the following preprocessing steps for work order data based on data type: performing spelling correction and standardization on the original text data; performing speech recognition on the original speech data using an industry-specific speech recognition model to convert it into corresponding text data; and performing image preprocessing on the original image data based on a visual encoder; while acquiring the work order data for the current work order, a globally unique identifier is generated for the current work order.

7. The intelligent customer service management method for work orders based on multimodal processing according to claim 6, characterized in that, The feature extraction steps specifically include: Step 1: The large language reasoning model, pre-trained based on the general language model, performs text encoding on the text data to extract deep semantic features, and performs visual extraction on the original image data based on the visual deep learning model or CLIP visual encoder to obtain visual features, thereby obtaining the text modality features and visual modality features of the current work. Step 2: Construct a multimodal contrastive loss function based on the contrastive learning strategy, and construct positive sample pairs and negative sample pairs based on the features of different modal data of the same work order. By minimizing the contrastive loss, feature vectors of different modalities of the same work order are made closer to each other in the embedding space, while semantic vectors of different work orders are made further apart. Step 3: Construct cross-modal semantic alignment through a cross-attention mechanism. Use text modality features as the query matrix and image modality features as the key and value matrices. Calculate cross-modal attention output to establish the correspondence between text descriptions and image regions, thereby achieving accurate matching between text and images. Then, fuse the text modality features and visual modality features of the current work order to obtain multimodal semantic features.

8. The intelligent customer service management method for work orders based on multimodal processing according to claim 1, characterized in that, The model reasoning steps specifically include: Step 1: Generating a query vector for the current work order based on its work order data; calculating the similarity between the query vector and each historical work order in the dynamic knowledge graph of the system to obtain multiple candidate historical work orders and constructing a first candidate set; querying the dynamic knowledge graph based on the entity and classification label identified by the current work order to obtain work order nodes directly connected to the entity of the current work order and work order nodes indirectly associated with the entity of the current work order, and then obtaining corresponding candidate historical work orders based on the work order nodes and constructing a second candidate set; merging and deduplicating the historical work orders in the first candidate set and the historical work orders in the second candidate set, calculating the comprehensive similarity of each candidate historical work order, and then obtaining multiple historical work orders similar to the current work order based on the comprehensive similarity and recording them as similar historical work orders; Step 2: Based on each similar historical work order and the dynamic knowledge graph, derive the solution, associated information graph profile, and process node knowledge for each similar historical work order, and form them into structured prompt words. These, along with the multimodal semantic feature vector of the current work order, are input into the context window of the large language reasoning model. This allows the large language reasoning model to generate the intent recognition and classification, key entity recognition and knowledge graph entity linking, and emergency procedure and impact scope determination results for the current work order based on the structured prompt words provided by the dynamic knowledge graph, thereby deriving the structured understanding result. The structured understanding result includes classification labels, priority scores, a list of associated knowledge graph entities, clusters of similar historical work orders, and possible root cause hypotheses inferred by the large language reasoning model.

9. The intelligent customer service management method for work orders based on multimodal processing according to claim 1, characterized in that, In the model reasoning step, when the large-scale reasoning model identifies new entities, new attributes, or newly inferred relationships during the reasoning process, it feeds back the current work order and the new entities, new attributes, or newly inferred relationships to the dynamic knowledge graph in real time, so as to construct new entity nodes, new attribute data, and new relationships within the dynamic knowledge graph. Specifically, when the large-scale language reasoning model identifies entities not stored in the dynamic knowledge graph, it generates candidate entity nodes based on the entities and infers the relationships between related entities based on the context to calculate the confidence score of the relationships. When the confidence score is greater than a preset threshold, the new entity is automatically written into the dynamic knowledge graph.

10. A work order intelligent customer service management system based on multimodal processing, characterized in that, The intelligent customer service management system for work orders includes: The data acquisition module is used to acquire the input data of the current work order and derive the work order data based on the input data of the work order data, and to perform corresponding preprocessing on the data of different modalities in the work order data. The feature extraction module is used to extract features from the preprocessed work order data through the multimodal encoder built by the system to obtain feature data of different modalities. It also combines contrastive learning to construct a multimodal contrastive loss function to map the feature data of different modalities to a unified semantic embedding space. At the same time, it establishes cross-modal semantic contrast through a cross-attention mechanism to fuse the feature data of different modalities into a multimodal semantic feature vector. The model reasoning module is used to input the multimodal semantic feature vector of the current work order into the pre-trained big language reasoning model of the system for reasoning. During the reasoning process, multiple historical work orders similar to the current work order are obtained from the dynamic knowledge graph constructed by the system and input into the context window of the big language reasoning model for interactive reasoning, thereby obtaining the structured understanding result of the current work order. The work order dispatch module is used to match the structured understanding result of the current work order with the customer service personnel in the system to determine the corresponding dispatcher and push the current work order to the corresponding dispatcher.

Images