Multi-role scene-oriented intelligent service verification system and differentiation evaluation method

Abstract

The invention provides a multi-role scene-oriented intelligent service verification system and a differentiation evaluation method, relates to the technical field of verification systems, and provides a multi-agent cooperation engine which is used for realizing cross-role dynamic task allocation and resource scheduling based on a reinforcement learning algorithm. The cross-framework model optimization layer is used for integrating an MXNet framework, a PyTorch framework and a TensorFlow framework and realizing model conversion through ONNX; the multi-mode interactive interface is used for supporting text, voice, image and video input; cross-device interaction of the mobile terminal and the intelligent loudspeaker box is adapted; a knowledge graph credibility evaluation module; by optimizing task allocation, model conversion, multi-modal interaction, knowledge graph credibility evaluation and the like, the performance and user experience of the intelligent service system are remarkably improved, meanwhile, a differentiated evaluation method is provided, and powerful support is provided for continuous optimization of the system.

Description

Technical Field

[0001] The present invention relates to the technical field of verification systems, and in particular to an intelligent service verification system and a differentiated evaluation method for multi-role scenarios. Background Art

[0002] In current intelligent service systems, especially in collaborative scenarios involving multiple roles, there are a series of significant problems and challenges. In traditional systems, task allocation between users, customer service representatives, and AI assistants relies heavily on fixed rules. For example, when a user inquires about product information at a store, the system may default to handing the inquiry over to a human customer service representative. However, response delays are common in e-commerce after-sales scenarios. In diverse business scenarios, the lack of compatibility between different devices and frameworks is also a prominent issue. Different scenarios may require the use of different deep learning frameworks, such as TensorFlow for model training, while PyTorch may be used when deployed through the model library. This can easily cause a large loss of accuracy during model conversion, especially in cases where hardware resources are limited, such as edge terminals. This problem is even more serious. For example, in mobile inference of medical imaging models, if a high-precision model is used directly, the return time of the result will be delayed by more than 2 seconds, seriously affecting the user experience.

[0003] Traditional intelligent service systems also suffer from the problem of incomplete understanding of the multimodal interaction context. Although text interaction is the mainstream form, their ability to capture user emotions (such as anxiety and dissatisfaction) and process multimodal input (such as voice and video) is still limited. For example, the recognition error rate of voice customer service systems in noisy environments is as high as 30%, and the accuracy of emotion recognition in video interactions is also less than 60%. This greatly affects the accuracy of service and user satisfaction. Therefore, it is particularly important to improve the accuracy of voice recognition and emotion recognition in order to provide more intimate and comprehensive services.

[0004] Therefore, we propose an intelligent service verification system and differentiated evaluation method for multi-role scenarios. Summary of the Invention

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: An intelligent service verification system for multi-role scenarios, including a multi-agent collaboration engine that implements dynamic cross-role task allocation and resource scheduling based on reinforcement learning algorithms; It is configured to achieve dynamic cross-role task allocation and resource scheduling based on reinforcement learning algorithms, including: optimizing task allocation paths based on historical data and real-time feedback, automatically determining request complexity and allocating them to the corresponding roles; Dynamically adjust the number of agents and task allocation algorithms during business peak periods; A cross-framework model optimization layer, which integrates MXNet, PyTorch, and TensorFlow frameworks and implements model conversion through ONNX. This layer uses dynamic quantization technology to compress model size, ensuring that the accuracy of the validation set does not drop by more than 2%. Migrating the teacher model to the student model through knowledge distillation improves reasoning speed; Multimodal interaction interface, which supports text, voice, image, and video input, including: combining ASR, TTS, and expression recognition technologies to achieve sentiment analysis and service strategy adjustment; Adapt to cross-device interaction of mobile terminals and smart speakers; The knowledge graph credibility assessment module builds a credible evidence model covering the entire knowledge life cycle, including: credibility assessment of data sources and extraction of only high-credibility data; The confidence of triples is calculated by comparing with authoritative data sources, low-confidence data is marked and manual review is triggered.

[0006] Preferably, in the e-commerce scenario, the multi-agent collaboration engine assigns simple order inquiries to the AI ​​assistant, and assigns complex order exceptions to manual customer service and synchronizes AI analysis results, thereby improving processing efficiency by reducing the number of manual transfers.

[0007] Preferably, the cross-framework model optimization layer: dynamic quantization technology enables the medical imaging model to adopt dynamic 8-bit quantization technology with an accuracy rate of ≥98.2%; Knowledge distillation technology reduces the inference latency of the customer service intent recognition model by 35%.

[0008] Preferably, the multimodal interactive interface in the travel platform: prioritizes anxious user requests based on expression recognition and voice analysis, improves itinerary change efficiency by 30%, and maintains the same session state between the smart speaker and the mobile phone terminal through a cross-device state synchronization protocol.

[0009] Preferably, the knowledge graph credibility assessment module in the epidemic monitoring scenario: The credibility of medical institution data sources is graded, and only high-level data is used to generate triples; When the confidence level of the number of confirmed cases is lower than the threshold, the health department will automatically trigger data verification.

[0010] A differentiated evaluation method for intelligent services in multi-role scenarios, including: a) Scenario prioritization; Prioritize optimizing intent recognition accuracy in high-frequency scenarios, and strengthen compliance verification in low-frequency, high-risk scenarios. The intent recognition response time for e-commerce after-sales consultation scenarios is ≤ 0.5 seconds; b) Guidance on model optimization direction; 200+ business scenario tags are added every month through user conversation log analysis; Automatic annotation of abnormal conversations shortens the knowledge base update cycle to 3 days; c) Human-machine collaboration efficiency evaluation; Quantify the complete context transfer rate during manual transfer, requiring the repeat inquiry rate to be less than 5%; Automatically associate historical consultation records with current symptoms in medical scenarios.

[0011] As a further improvement of the above technical solution: The evaluation method further comprises: d) Intent recognition accuracy evaluation; Distinguish explicit demands from implicit compensation claims, and maintain cross-round context memory ≥ 90%; Verify the accuracy of implicit intent recognition through a closed-loop user feedback loop; e) Dynamic scenario stress testing; Simulate price anomaly inquiries during a major promotion, requiring the scenario knowledge base to respond in less than 1 second. Concurrent request processing capacity ≥ 10,000 times / minute; f) Multimodal processing capability assessment; ASR recognition rate in quiet environment ≥95%, error rate in noisy environment ≤18%; The signal-to-noise ratio of the noise suppression algorithm in voice customer service scenarios is improved by ≥15dB.

[0012] Preferably, the low-frequency, high-risk scenarios include financial compliance consulting, and the evaluation criteria are: 100% compliance verification coverage of multiple rounds of dialogue; manual review trigger conditions include sensitive word matching and logical contradiction detection.

[0013] Compared with the prior art, the present invention has the following beneficial effects: Dynamic task allocation and resource scheduling optimization across roles: Through a multi-agent collaboration engine and reinforcement learning algorithm, the system can optimize task allocation paths based on historical data and real-time feedback, automatically determine request complexity and assign them to corresponding roles, thereby significantly improving processing efficiency and reducing the number of manual transfers.

[0014] Cross-framework model optimization: The cross-framework model optimization layer integrates multiple deep learning frameworks (such as MXNet, PyTorch, and TensorFlow) and implements model conversion through ONNX, solving compatibility issues between different devices and frameworks. Dynamic quantization technology and knowledge distillation technology are used to effectively compress the model size and improve the inference speed while maintaining high accuracy, especially when hardware resources are limited (such as edge terminals).

[0015] Improved multimodal interaction capabilities: The multimodal interaction interface supports multiple input methods such as text, voice, image, and video. Combined with ASR, TTS, and expression recognition technologies, it enables more comprehensive sentiment analysis and service strategy adjustments.

[0016] It is compatible with cross-device interactions such as mobile terminals and smart speakers, improving service convenience and user experience.

[0017] Knowledge graph credibility assessment: Build a trusted evidence model covering the entire life cycle of knowledge, conduct credibility assessments on data sources, and only extract high-credibility data to ensure the accuracy and reliability of knowledge. Calculate triple confidence by comparing with authoritative data sources, mark low-confidence data, and trigger manual review, further improving the credibility of knowledge.

[0018] Differentiated evaluation methods: This includes scenario prioritization, model optimization guidance, human-machine collaboration efficiency assessment, intent recognition accuracy assessment, dynamic scenario stress testing, and multimodal processing capability assessment, providing clear direction and standards for continuous system optimization. In particular, the assessment criteria for low-frequency, high-risk scenarios (such as financial compliance consulting) ensure the compliance and security of the system in these key areas.

[0019] In summary, the present invention significantly improves the performance and user experience of the intelligent service system by optimizing task allocation, model conversion, multimodal interaction, and knowledge graph credibility evaluation, while providing a differentiated evaluation method, providing strong support for the continuous optimization of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a system block diagram of the intelligent service verification system for multi-role scenarios provided by the present invention. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] To facilitate understanding of the present invention, the present invention will be described more comprehensively below with reference to relevant references, and several embodiments of the present invention are given. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0023] It should be noted that when an element is referred to as being "fixed on" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used in this article are for illustrative purposes only.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items. Example

[0025] like Figure 1 As shown, the intelligent service verification system for multi-role scenarios of this embodiment includes: The multi-agent collaboration engine is used to implement dynamic cross-role task allocation and resource scheduling based on reinforcement learning algorithms. Specifically, its configuration is as follows: Task allocation path optimization: Based on a preset reinforcement learning algorithm, using historical data and real-time feedback information, the task allocation path is optimized through a specific optimization algorithm (such as Q-learning, policy gradient algorithm, etc., which can be selected according to actual conditions); Request complexity assessment and allocation: A preset complexity assessment model (e.g., a machine learning-based classification model, such as a decision tree classification model) is used to automatically determine request complexity and assign requests to corresponding roles based on preset rules. For example, allocation rules can be formulated based on factors such as the depth of expertise required for the request and the estimated processing time. Peak-period adjustments: During peak periods, the number of agents is dynamically adjusted based on real-time business load conditions through preset dynamic adjustment strategies (such as threshold-based adjustment strategies, which trigger adjustments when the business load exceeds a preset threshold). The system also switches to a task allocation algorithm suitable for peak periods (such as an improved ant colony algorithm, which can quickly find a more optimal task allocation solution based on real-time task distribution). The specific application of the multi-agent collaboration engine in e-commerce scenarios is: based on preset order complexity judgment rules (such as judgment based on order amount, product type, historical abnormal records, etc.), simple order queries are assigned to AI assistants for processing, and complex order exceptions are assigned to manual customer service, and AI analysis results (such as potential risk points of orders, relevant historical data, etc.) are synchronized during the assignment process; through the above task allocation strategy, the number of manual transfers is reduced, and compared with existing technologies, the order processing efficiency is improved by [X]%.

[0026] Cross-framework model optimization layer A cross-framework model optimization layer, which integrates MXNet, PyTorch, and TensorFlow frameworks and implements model conversion through ONNX. Specifically, it includes: Model quantization and compression: Dynamic quantization technology (such as dynamic 8-bit quantization technology, which can effectively reduce the model size without significantly reducing model performance) is used to quantize and compress the model. During the quantization process, the accuracy of the validation set is monitored to ensure that the accuracy drop does not exceed 2%.

[0027] Knowledge distillation optimization: Using knowledge distillation technology (such as feature distillation, response distillation, and other methods, for example, using intermediate-layer feature distillation to transfer the feature information of the intermediate layer of the teacher model to the student model), the knowledge of the trained teacher model is transferred to the student model to improve the inference speed of the student model.

[0028] The effect of the cross-framework model optimization layer in specific scenarios is as follows: dynamic 8-bit quantization technology is used to quantize the medical imaging model. The accuracy of the quantized medical imaging model on the test set is ≥98.2%. Compared with the model before quantization, the model size is compressed while ensuring accuracy; knowledge distillation technology is used to optimize the customer service intent recognition model. Under the same hardware environment, the inference delay of the optimized customer service intent recognition model is reduced by 35% compared with the pre-optimization model.

[0029] Multimodal interactive interface Multimodal interaction interface, which supports text, voice, image, and video input, including: Sentiment analysis and service strategy adjustment: Automatic speech recognition (ASR) technology is used to convert voice input into text, and text-to-speech (TTS) technology is used to achieve voice output. In addition, expression recognition technology (such as deep learning-based expression recognition models, such as convolutional neural network (CNN) models) is used to analyze user emotions. Service strategies are adjusted based on the sentiment analysis results (such as adjusting the tone of responses and providing more detailed solutions). Cross-device interaction: By developing a unified cross-device interaction protocol that adapts to different devices such as mobile terminals and smart speakers, seamless interaction between different devices is achieved, including synchronization of session states (for example, smart speakers and mobile terminals maintain the same session state). The specific application of multimodal interactive interface in tourism platform is: based on facial expression recognition technology (such as identifying the user's frown, anxious eyes and other facial features) and voice analysis technology (such as analyzing the voice intonation, speaking speed, keywords, etc.), it determines whether the user is in an anxious state, and gives priority to the requests of anxious users (such as shortening the response time, providing more detailed services, etc.). Through the above-mentioned strategy of prioritizing the requests of anxious users, compared with the existing technology, the efficiency of itinerary changes on the tourism platform is improved by 30%. A cross-device status synchronization protocol is formulated and implemented to ensure that smart speakers and mobile terminals can maintain the same session state when users perform operations such as itinerary changes, thereby achieving seamless interaction.

[0030] Knowledge graph credibility assessment module The knowledge graph credibility assessment module is used to build a credible evidence model covering the entire knowledge life cycle, specifically including: Data source credibility assessment and extraction: Use preset credibility assessment indicators (such as the authority of the data source, data update frequency, etc.) to assess the credibility of the data source. Based on the assessment results, only data with credibility above the preset threshold is extracted; Triple confidence calculation and review: The extracted data is compared with the authoritative data source, and the confidence of the triple is calculated using a specific confidence calculation method (such as similarity calculation, statistical method, etc.). Triples with confidence below the preset threshold are marked and the manual review process is triggered; The specific application of the knowledge graph credibility assessment module in the epidemic monitoring scenario is as follows: using preset credibility grading standards (such as based on the qualifications of the medical institution, the standardization of data collection, etc.) to grade the credibility of medical institution data sources, only using data with a credibility level higher than the preset level to generate triplets in the knowledge graph, and setting a confidence threshold for the number of confirmed cases. When the confidence level of the number of confirmed cases in the generated triplet is lower than the threshold, the data verification process with the health department is automatically triggered to ensure the accuracy and reliability of the data; The differentiated evaluation method for intelligent services in multi-role scenarios in this embodiment includes: a) Scenario Prioritization: Based on the frequency of use and risk level, scenarios are divided into high-frequency scenarios and low-frequency, high-risk scenarios. For high-frequency scenarios, prioritize optimizing intent recognition accuracy. For low-frequency, high-risk scenarios, strengthen compliance verification (such as through multi-round dialogue compliance verification and sensitive word detection). In e-commerce after-sales consultation scenarios, the intent recognition response time is required to be ≤ 0.5 seconds to ensure user experience.

[0031] b) Model Optimization Guidance: Utilize natural language processing technology and data analysis methods to analyze user conversation logs, adding at least 200 business scenario tags each month to enrich the knowledge base. Establish an automatic annotation mechanism for anomalous conversations, automatically labeling them using pre-set rules and models. Update the knowledge base based on the annotation results, shortening the knowledge base update cycle to 3 days.

[0032] c) Human-machine collaboration efficiency assessment: Develop quantitative metrics for context completeness. When users transition from intelligent services to human customer service, ensure that context completeness meets preset standards. At the same time, maintain a repeat inquiry rate of less than 5% to improve human-machine collaboration efficiency. In medical scenarios, by establishing a database of user historical consultation records and a symptom association model, users' historical consultation records are automatically linked to their current symptoms, providing doctors with a more comprehensive basis for diagnosis.

[0033] Assessment methods also include: d) Intent recognition accuracy assessment: Use specific algorithms or models (such as those based on semantic analysis and contextual understanding) to distinguish between users' explicit demands and implicit compensation claims. During multi-round conversations, ensure that the cross-round context memory retention rate is ≥ 90% to ensure the accuracy of intent recognition. Establish a user feedback closed-loop mechanism to collect user feedback on implicit intent recognition results. Verify the accuracy of implicit intent recognition by comparing the feedback information with the recognition results, and optimize the recognition model based on the verification results.

[0034] e) Dynamic Scenario Stress Testing: This simulates price anomaly inquiries that might occur during a major sales event. Using stress testing tools, we simulate a large number of concurrent requests. The scenario knowledge base is required to handle these requests with a response time of less than 1 second to ensure system stability under high concurrency. During dynamic scenario stress testing, the system is required to handle ≥ 10,000 concurrent requests per minute to meet actual business needs.

[0035] f) Multimodal processing capability assessment: In a quiet environment, a standard speech test set is used for testing, requiring the automatic speech recognition (ASR) recognition rate to be ≥95%. In a noisy environment, by simulating scenarios with different noise levels, the ASR error rate is required to be ≤18% to evaluate the speech recognition performance of multimodal processing capabilities. In the voice customer service scenario, a noise suppression algorithm is used to process the voice signal. By comparing the signal-to-noise ratio before and after processing, the signal-to-noise ratio is required to be improved by ≥15dB to improve the quality of voice customer service.

[0036] Low-frequency, high-risk scenarios include financial compliance consulting. The assessment criteria for this scenario are: Comprehensive compliance verification is conducted across multiple rounds of conversations in financial compliance consulting, ensuring 100% compliance verification coverage to ensure that the consulting process complies with relevant laws, regulations, and regulatory requirements. Manual review triggers are set. When sensitive word matches (such as those involving illegal financial activities and illegal operations) or logical contradictions (such as inconsistencies in previous statements or violations of financial common sense) occur in multiple rounds of conversations, the manual review process is automatically triggered.

[0037] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent service verification system for multi-role scenarios, characterized by: include: A multi-agent collaboration engine, which is used to implement dynamic cross-role task allocation and resource scheduling based on reinforcement learning algorithms; It is configured to achieve dynamic cross-role task allocation and resource scheduling based on reinforcement learning algorithms, including: optimizing task allocation paths based on historical data and real-time feedback, automatically determining request complexity and allocating them to the corresponding roles; Dynamically adjust the number of agents and task allocation algorithms during business peak periods; A cross-framework model optimization layer, which integrates MXNet, PyTorch, and TensorFlow frameworks and implements model conversion through ONNX. This layer uses dynamic quantization technology to compress model size, ensuring that the accuracy of the validation set does not drop by more than 2%. Migrating the teacher model to the student model through knowledge distillation improves reasoning speed; Multimodal interaction interface, which supports text, voice, image, and video input, including: combining ASR, TTS, and expression recognition technologies to achieve sentiment analysis and service strategy adjustment; Adapt to cross-device interaction of mobile terminals and smart speakers; The knowledge graph credibility assessment module builds a credible evidence model covering the entire knowledge life cycle, including: credibility assessment of data sources and extraction of only high-credibility data; The confidence of triples is calculated by comparing with authoritative data sources, low-confidence data is marked and manual review is triggered.

2. The system according to claim 1, wherein: In e-commerce scenarios, the multi-agent collaboration engine assigns simple order queries to AI assistants, and complex order exceptions to manual customer service and synchronizes AI analysis results, thereby improving processing efficiency by reducing the number of manual transfers.

3. The system according to claim 1, wherein: The cross-framework model optimization layer: dynamic quantization technology enables the medical imaging model to use dynamic 8-bit quantization technology with an accuracy rate of ≥ 98.2%; Knowledge distillation technology reduces the inference latency of the customer service intent recognition model by 35%.

4. The system according to claim 1, wherein: The multimodal interaction interface is used in travel platforms to prioritize anxious user requests based on expression recognition and voice analysis, improving itinerary change efficiency by 30%. Through a cross-device state synchronization protocol, the smart speaker and mobile phone terminal maintain the same session state.

5. The system according to claim 1, wherein: The knowledge graph credibility assessment module is used in the epidemic monitoring scenario: The credibility of medical institution data sources is graded, and only high-level data is used to generate triples; When the confidence level of the number of confirmed cases is lower than the threshold, the health department will automatically trigger data verification.

6. A differentiated evaluation method for intelligent services in multi-role scenarios, characterized by: Assessment methods include: a) Scenario prioritization; Prioritize optimizing intent recognition accuracy in high-frequency scenarios, and strengthen compliance verification in low-frequency, high-risk scenarios. The intent recognition response time for e-commerce after-sales consultation scenarios is ≤ 0.5 seconds; b) Guidance on model optimization direction; 200+ business scenario tags are added every month through user conversation log analysis; Automatic annotation of abnormal conversations shortens the knowledge base update cycle to 3 days; c) Human-machine collaboration efficiency evaluation; Quantify the complete context transfer rate during manual transfer, requiring the repeat inquiry rate to be less than 5%; Automatically associate historical consultation records with current symptoms in medical scenarios.

7. The method according to claim 6, characterized in that The evaluation method further comprises: d) Intent recognition accuracy evaluation; Distinguish explicit demands from implicit compensation claims, and maintain cross-round context memory ≥ 90%; Verify the accuracy of implicit intent recognition through a closed-loop user feedback loop; e) Dynamic scenario stress testing; Simulate price anomaly inquiries during a major promotion, requiring the scenario knowledge base to respond in less than 1 second. Concurrent request processing capacity ≥ 10,000 times / minute; f) Multimodal processing capability assessment; ASR recognition rate in quiet environment ≥95%, error rate in noisy environment ≤18%; The signal-to-noise ratio of the noise suppression algorithm in voice customer service scenarios is improved by ≥15dB.

8. The method according to claim 6, characterized in that The low-frequency, high-risk scenarios include financial compliance consulting, and their evaluation criteria are: 100% compliance verification coverage of multiple rounds of dialogue; manual review trigger conditions include sensitive word matching and logical contradiction detection.

Images