Virtual digital outbound intelligent customer service system based on generative AI

Abstract

The invention discloses a virtual digital outbound intelligent customer service system based on generative AI, relates to the technical field of virtual customer service management, and solves the technical problems that the system lacks an effective monitoring mechanism for self service quality and has a single encryption processing mode for interactive data. According to the method, the overall state of the intelligent customer service is monitored in real time by calculating multi-dimensional indexes such as the knowledge anchor-free answer rate, the semantic ambiguity answer proportion and the high-frequency complaint question association rate, management personnel can conveniently optimize a knowledge base, an answer template or a service process in a targeted mode, the user service experience and the system reliability are improved, and the user experience is improved. A random modification feature is generated according to the byte size of a data packet, the data packet is dynamically bundled and sent out of order, and a random delay transmission strategy is introduced, so that analysis of an attacker on traffic features is effectively interfered, data eavesdropping and content tampering are prevented, and the security and privacy of user interaction information in the transmission process are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual customer service management, and specifically to a virtual digital outbound intelligent customer service system based on generative AI. Background Art

[0002] With the rapid development of artificial intelligence technology, virtual digital outbound intelligent customer service systems have been widely used in many fields such as finance, e-commerce, and government affairs.

[0003] Traditional intelligent customer service systems rely on fixed rules or simple machine learning models, enabling human-computer interaction through pre-set script libraries. Speech recognition technology primarily combines traditional acoustic and language models. However, in scenarios with complex accents and background noise, speech-to-text conversion accuracy is low. Furthermore, semantic understanding struggles with complex questions involving ambiguous semantics and contextual dependencies, resulting in customer service system responses lacking pertinence and accuracy.

[0004] In terms of data transmission, the existing system's single encryption method for interactive data makes it easy for attackers to obtain sensitive information through traffic analysis and packet eavesdropping. Furthermore, the system lacks an effective mechanism for monitoring its own service quality, making it unable to promptly detect issues such as unfounded answers and ambiguous semantics, making it difficult to ensure user experience and service reliability. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a virtual digital outbound intelligent customer service system based on generative AI, which solves the problems of the system's lack of an effective monitoring mechanism for its own service quality and a single encryption processing method for interactive data.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a virtual digital outbound intelligent customer service system based on generative AI, comprising:

[0007] The speech conversion and recognition module is used to convert, recognize and analyze the interactive information transmitted by the interactive information collection module to obtain text semantics, and determine the joint confidence by calculating the confidence of the text semantics and semantic information. At the same time, it matches the threshold interval to generate interactive response information and transmits it to the intelligent customer service comprehensive judgment module and the interactive transmission analysis module;

[0008] The intelligent customer service comprehensive judgment module is used to combine interactive response information and historical interaction records to conduct multi-dimensional indicator monitoring and analysis. It calculates the knowledge-free anchor answer rate, the proportion of semantically ambiguous answers, and the correlation rate of high-frequency complaint issues. It then uses these three factors to judge the overall status of the intelligent customer service, generates normal or abnormal status information, and transmits it to the intelligent management output module.

[0009] The interactive transmission analysis module is used to perform transmission encryption analysis on the obtained interactive response information, split the interactive response information into independent data packets, obtain data packet characteristics, and generate random byte segments based on the byte tail values ​​of the data packets, and combine them to obtain a combined data packet;

[0010] Then the combined data packets are bundled and grouped and transmitted in a disorderly order and random delay manner to generate interactive transmission information and transmit it to the intelligent management output module.

[0011] As a further solution of the present invention, it also includes an interactive information collection module and an intelligent management output module;

[0012] The interaction information acquisition module is used to obtain the interaction information, which is voice information, and transmit it to the voice conversion and recognition module;

[0013] The intelligent management output module is used to display normal or abnormal status information and interactive transmission information to the corresponding management personnel.

[0014] As a further solution of the present invention, the specific method for the speech conversion recognition module to obtain text semantics is:

[0015] Acquire interactive information and convert it into text information through speech recognition technology. Simultaneously, use natural language processing technology to identify the obtained text information to obtain text semantics. Then, use text similarity algorithm to remove duplicate corpus in the text semantics and generate preprocessed semantics.

[0016] Obtain preprocessed semantics, combine them with non-voice data from customer service scenarios, and align the preprocessed semantics with business data through a multimodal model. At the same time, calculate the confidence of text semantics and voice information separately.

[0017] As a further solution of the present invention, the speech conversion recognition module calculates the text semantic confidence and the speech information confidence respectively in the following specific manners:

[0018] Calculate the text semantic confidence, process the audio by ASR model, obtain the output character probability of each frame, and then calculate the joint product of all character probabilities. Calculate the text semantic confidence ASR, where N is the number of frames, specifically i = 1, 2, ..., N, P i，j is the probability of recognizing character j in the i-th frame, and the average of the maximum probability of each frame is taken;

[0019] Calculate the confidence of voice information, build the "acoustic feature-business semantics" matching model, extract the correlation between voice prosodic features and semantic intent, and use the formula The matching degree is calculated, and then the confidence level corresponding to the voice information is calculated according to the formula confidence level = matching degree × weight.

[0020] As a further solution of the present invention, the specific manner in which the speech conversion recognition module generates interactive response information is as follows:

[0021] According to formula C z =α×ASR+(1-α)×P to calculate the joint confidence C z , where α is the text semantic confidence weight, P represents the voice information confidence, and the obtained joint confidence C z Match the corresponding threshold interval, determine the corresponding response strategy and generate interactive response information. The specific matching method is as follows:

[0022] If the joint confidence C z ≥0.85, the corresponding response strategy is to directly output the answer;

[0023] If 0.6≤joint confidence C z <0.85, then the corresponding response strategy is to trigger further questions for clarification;

[0024] If the joint confidence C z <0.6, the corresponding response strategy is to transfer to manual or refuse to answer.

[0025] As a further solution of the present invention, the intelligent customer service comprehensive judgment module calculates the knowledge-free anchor answer rate, the semantically ambiguous answer ratio, and the high-frequency complaint question correlation rate in the following specific ways:

[0026] Calculate the rate of no-knowledge anchor answer, set the time period T, obtain all answer texts within the period, and use the semantic similarity algorithm to match the standard answer of the knowledge base to screen out answers with similarity less than the threshold value as no-knowledge anchor answer. According to the formula Calculate the knowledge-free anchor answer rate and compare it with the preset anchor answer rate. If it is lower than the preset value, it is normal; otherwise, it is abnormal.

[0027] Calculate the proportion of semantically ambiguous answers. Use the ambiguity detection model, input the answer text, output the ambiguity probability, and if the ambiguity probability is ≥ 0.5, it is determined to be ambiguous. Count the number of ambiguous answers and calculate the answer according to the formula Calculate the proportion of semantically ambiguous answers and compare it with the preset value of ambiguous answer proportion. If it is lower than the preset value, it is normal; otherwise, it is abnormal.

[0028] Calculate the correlation rate of high-frequency complaint issues. Based on historical complaint data, count the top K issues with the highest number of complaints to build a keyword library. The value of K is set by the operator. Perform keyword matching + semantic vector similarity detection on the consultation text to identify related issues. According to the formula The correlation rate of high-frequency complaint issues is calculated and compared with the preset correlation rate value. If it is higher than the preset value, it is abnormal, otherwise it is normal.

[0029] As a further solution of the present invention, the specific manner in which the intelligent customer service comprehensive judgment module generates normal or abnormal status information is as follows:

[0030] A judgment is made based on the analysis results of the answer rate without knowledge anchor points, the proportion of semantically ambiguous answers, and the correlation rate of high-frequency complaint issues. If any group of anomalies exists, it means that the overall status of the intelligent customer service is abnormal, and status adjustment information is generated. Conversely, if all are normal, it means that the overall status of the intelligent customer service is normal, and normal status information is generated.

[0031] As a further solution of the present invention, the specific manner in which the interactive transmission analysis module obtains the combined data packet is:

[0032] Obtain interactive response information, split it into independent data packets and label them as n, and n=1, 2, ..., m, where m represents the number of data packets. At the same time, obtain its corresponding data packet characteristics, obtain the byte size of the data packet, and obtain its byte tail value. At the same time, randomly generate byte segments with double the byte tail value as the standard byte number, and add the generated byte segments to the data packet characteristics to obtain a combined data packet.

[0033] As a further solution of the present invention, the specific manner in which the interactive transmission analysis module generates the interactive transmission information is as follows:

[0034] All combined data packets are obtained and h groups are randomly selected to be bundled to obtain a bundled data packet. All bundled data packets are obtained by analogy and sent out of order. At the same time, random delay is used for periodic interval transmission and interactive transmission information is generated.

[0035] The present invention provides a virtual digital outbound intelligent customer service system based on generative AI. Compared with the existing technology, it has the following advantages:

[0036] This invention leverages a Transformer-based industrial-grade ASR model and pre-trained NLP models like BERT, combined with multimodal data and CLIP-like multimodal frameworks, to achieve high-precision voice conversion and semantic understanding, enabling precise identification of user intent. By calculating the confidence level of both text semantics and voice information, and performing a comprehensive assessment, the system matches response strategies based on the combined confidence levels, ensuring the accuracy and pertinence of interactive responses.

[0037] This invention generates random modification features for packet byte size, dynamically bundles packets, and sends them out of order. It also introduces a randomized transmission delay strategy, effectively interfering with attackers' analysis of traffic characteristics, preventing data eavesdropping and content tampering. It ensures the security and privacy of user interaction information during transmission. By calculating multi-dimensional indicators such as the rate of answers without knowledge anchors, the proportion of semantically ambiguous answers, and the correlation rate of high-frequency complaints, it monitors the overall status of intelligent customer service in real time. Once an abnormal indicator is detected, status adjustment information is generated in a timely manner, allowing managers to optimize the knowledge base, answer templates, or service processes in a targeted manner, improving the user service experience and system reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a system block diagram of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0040] Example 1

[0041] See also Figure 1 This application provides a virtual digital outbound intelligent customer service system based on generative AI, including: interactive information collection module, voice conversion recognition module, intelligent customer service comprehensive judgment module, interactive transmission analysis module and intelligent management output module, and combined with Figure 1 It can be known that the functional modules are electrically connected in a unidirectional manner.

[0042] The interactive information acquisition module is used to obtain interactive information and transmit it to the speech conversion and recognition module. The interactive information obtained here is mainly speech information.

[0043] The speech conversion and recognition module is used to convert and analyze the obtained interactive information, converting it into text information through speech recognition technology. Relying on mature ASR models (such as industrial-grade solutions based on Transformer), it converts customer service calls and user consultation voices into text frame by frame. At the same time, it uses natural language processing (NLP) technology to identify the text information obtained and obtain text semantics. Pre-trained models such as BERT are used to parse the text semantics and identify the core intent. A text similarity algorithm is used to remove duplicate corpus in the text semantics to generate preprocessed semantics.

[0044] Next, we obtain preprocessed semantics and combine them with non-speech data such as text logs and user profiles from customer service scenarios. We then align the preprocessed semantics with the business data using a multimodal model, and calculate the confidence of the text semantics and the confidence of the speech information.

[0045] Integrate non-voice data such as customer service text logs (historical work orders, FAQs), user profiles (membership levels, purchase records), and build a multimodal semantic alignment model:

[0046] Data association: Use the user order number as the association key to match pre-processed semantics with business data (such as the number of historical returns by the user and whether the product complies with the return and exchange policy).

[0047] Multimodal calibration: Through the CLIP-type multimodal framework, semantic text and business data features are aligned (such as mapping the "return" semantics with the "after-sales process" knowledge base vector) to ensure that semantic understanding is consistent with customer service business rules.

[0048] Calculate the semantic confidence of the text, process the audio by framing it through the ASR model (speech to text), and obtain the probability of outputting characters in each frame. Then calculate the joint product of all character probabilities, according to the formula Calculate the text semantic confidence ASR, where N is the number of frames, specifically i = 1, 2, ..., N, P i，j is the probability of recognizing character j in the i-th frame, and the average of the maximum probability of each frame is taken;

[0049] For example, if a speech segment is divided into 100 frames and the average maximum character probability in each frame is 0.92, then the ASR confidence level is 0.92, which represents the basic reliability of speech-to-text conversion.

[0050] Calculate the confidence of voice information, build the "acoustic feature-business semantics" matching model, extract the correlation between voice prosodic features (speech speed, pauses, emotional tone) and semantic intent, and use the formula The matching degree is calculated, and then the confidence level corresponding to the voice information is calculated according to the formula: confidence level = matching degree × weight;

[0051] For example, if a user angrily demands a return, the "rapid, high-frequency" acoustic features in the voice match the "urgent after-sales" business semantics at a rate of 0.85, and the business weight is 0.7. Therefore, the voice confidence is 0.85×0.7=0.595.

[0052] The obtained text semantic confidence ASR and voice information confidence are calculated comprehensively, according to formula C z =α×ASR+(1-α)×P to calculate the joint confidence C z , where α is the text semantic confidence weight, P represents the voice information confidence, and the obtained joint confidence C zMatch the corresponding threshold interval, determine the corresponding response strategy and generate interactive response information. The specific matching method is as follows:

[0053] If the joint confidence C z ≥0.85, the corresponding response strategy is to directly output the answer;

[0054] If 0.6≤joint confidence C z <0.85, then the corresponding response strategy is to trigger further questions for clarification;

[0055] If the joint confidence C z <0.6, the corresponding response strategy is to transfer to manual or refuse to answer.

[0056] At the same time, the generated interactive response information is transmitted to the intelligent customer service comprehensive judgment module and the interactive transmission analysis module.

[0057] For example, when a user says, "The charging port on my phone is broken," after speech-to-text conversion, the BERT model identifies the intent as "after-sales repair" (with a probability of 0.78) and the entity as "phone charging port" (with a probability of 0.92).

[0058] After 50 samplings through MCDropout, the variance of the intent probability is 0.05 (low variance, increased confidence);

[0059] The knowledge graph verifies that "charging port damage" is covered by the warranty, with a matching degree of 100% and a corrected confidence of 0.78×1.1=0.858. Combined with the corresponding matching criteria, the response strategy can be determined as "directly reply to the repair process."

[0060] The intelligent customer service comprehensive judgment module is used to analyze the overall status of the intelligent customer service based on the obtained interactive response information, obtain the corresponding historical interaction records of the intelligent customer service, and conduct multi-dimensional indicator monitoring and analysis based on the historical interaction records. The multi-dimensional indicators specifically include the knowledge-free anchor point answer rate, the proportion of semantically ambiguous answers, and the correlation rate of high-frequency complaint issues;

[0061] Calculate the rate of no-knowledge anchor answers, obtain all answer texts within the time period T in the historical interaction records, and the specific value of the time period T is set by the operator, match them with the standard answers in the knowledge base through the semantic similarity algorithm (such as BERT+cosine similarity), set the threshold (such as similarity ≥ 0.7 is determined to have an anchor point), and filter out answers with similarity < 0.7 as "no-knowledge anchor answers", and then use the formula Calculate the no-knowledge anchor response rate;

[0062] At the same time, the obtained no-knowledge anchor answer rate is compared with the anchor answer rate preset value, and the specific value of the anchor answer rate preset value is set by the operator. If the no-knowledge anchor answer rate is less than the anchor answer rate preset value, it means that the no-knowledge anchor answer rate is normal, otherwise it means that the no-knowledge anchor answer rate is abnormal;

[0063] For example, a customer service system processes 1,000 inquiries a day, of which 850 answers have a similarity ≥ 0.7 with the knowledge base answers, and 150 are unmatched. The corresponding no-knowledge anchor answer rate is then calculated to be 15%. The preset anchor answer rate is further obtained as 5%. After comparison, it can be known that the current no-knowledge anchor answer rate is abnormal.

[0064] Calculate the proportion of semantically ambiguous answers. Use an ambiguity detection model (such as a semantic multi-solution classifier based on LSTM) to input the answer text and output whether it is ambiguous (probability ≥ 0.5 is considered ambiguous). Perform ambiguity detection on each answer, count the number of ambiguous answers, and use the formula The proportion of semantically ambiguous answers is calculated and compared with the preset value of the proportion of ambiguous answers. If the proportion of semantically ambiguous answers is greater than the preset value, it means that the proportion of semantically ambiguous answers is abnormal; otherwise, it means that the proportion of semantically ambiguous answers is normal.

[0065] For example, when a customer service representative responds, “Your application will take approximately 3 working days to process,” the word “approximately” may be misunderstood by the user as “within 3 days” or “more than 3 days,” and the model will identify it as ambiguous.

[0066] If 50 ambiguous answers are detected among 1,000 answers, accounting for 5%, and the preset value of the proportion of ambiguous answers is 8%, then after comparison, it can be known that the proportion of semantically ambiguous answers is normal.

[0067] Calculate the association rate of high-frequency complaint issues. Based on historical complaint data, count the top K issues in terms of complaint volume (such as "slow refund" and "product quality issues") to form a keyword / semantic vector library. Here, K represents the ranking, and the specific value is set by the operator. The consultation text contains complaint library keywords (such as "refund" and "quality"), which are initially marked as associated. Through topic models (such as LDA) or semantic vector similarity, identify implicit associations (such as the semantic association between "funds not received" and "slow refund"). Then, according to the formula Calculate the correlation rate of high-frequency complaint issues and compare it with the preset correlation rate value;

[0068] If the correlation rate of high-frequency complaint issues is greater than the preset correlation rate value, it means that the correlation rate of high-frequency complaint issues is abnormal. Conversely, if the correlation rate of high-frequency complaint issues is less than the preset correlation rate value, it means that the correlation rate of high-frequency complaint issues is normal.

[0069] For example, historically high-frequency complaints include "refund delays" (keywords: refund, delay, arrival of funds), and a consultation text is "I have submitted a refund application for 3 days but have not received the money yet", which is determined to be related through keyword + semantic matching.

[0070] If 80 out of 1,000 consultations are related to high-frequency complaint issues, the correlation rate of high-frequency complaint issues is 8%, and the preset value of the correlation rate is 10%, after comparison, it can be known that the correlation rate of high-frequency complaint issues is less than the preset value of the correlation rate, then the correlation rate of high-frequency complaint issues is normal.

[0071] The system makes a judgment based on the analysis results of the answer rate without knowledge anchor points, the proportion of semantically ambiguous answers, and the correlation rate of high-frequency complaint issues. If any of the above abnormalities exists, it means that the overall status of the intelligent customer service is abnormal, and status adjustment information is generated. Otherwise, if everything is normal, it means that the overall status of the intelligent customer service is normal, and normal status information is generated. Both are transmitted to the intelligent management output module at the same time.

[0072] Intelligent management output module, which is used to display the acquired normal or abnormal status information to the corresponding management personnel.

[0073] Example 2

[0074] As the second embodiment of the present invention, the focus is on combining the implementation processes of the first and second embodiments.

[0075] The interactive transmission analysis module is used to obtain the interactive response information transmitted by the semantic transformation identification module and perform transmission encryption analysis on it. The interactive response information is obtained and split into independent data packets and labeled as n, and n = 1, 2, ..., m, where m represents the number of data packets. At the same time, the corresponding data packet features are obtained, and the data packet features here include timestamps or protocol header fields. The key features of each data packet are extracted, including timestamps (recording the time when the data packet is generated) and protocol header fields (such as source port, destination port, sequence number, etc. in TCP / UDP protocol). Taking a data packet based on the TCP protocol as an example, its source port 50001, destination port 8080 and sequence number 123456 are extracted, and random modification features are generated at the sending end based on the byte size of the data packet. The generation method is as follows:

[0076] Get the byte size of the data packet and its byte mantissa value. At the same time, randomly generate a byte segment with double the byte mantissa value as the standard byte number. Add the generated byte segment to the data packet feature to obtain a combined data packet. Specifically, if the mantissa value is 4, the generated byte segment length is 8. If the mantissa value is 9, the generated byte segment length is 18. If the mantissa value is 0, a byte segment length of 20 is generated. For example, if the original data packet timestamp is 2025-06-17 10:30:00, it becomes 2025-06-17 10:30:00 + random byte segment after merging.

[0077] Obtain all combined data packets and randomly select h groups to bundle to obtain bundled data packets. The specific value of h here is set by the operator and is in the range of 2-9. Similarly, obtain all bundled data packets and send the obtained bundled data packets in random order. At the same time, use random delay to transmit periodically, insert random delay at the session layer: randomly wait 0-50ms before sending (need to balance delay and call experience), destroy the "periodic packet sending" feature, and use group random order sending: divide continuous voice packets into small groups (such as 5 packets per group), randomly adjust the sending order within the group, restore it at the receiving end, and generate interactive transmission information. Specifically, the corresponding transmission decryption rules will be generated at the receiving end, and the obtained interactive transmission information will be reversely decrypted according to the transmission decryption rules to obtain the corresponding interactive response information, which will then be transmitted to the intelligent management output module.

[0078] Assume there are 10 combined data packets in total, and set h = 3. Then the 1st, 2nd, and 3rd combined data packets are bundled into one group, the 4th, 5th, and 6th are bundled into another group, and so on. The original order is 1, 2, and 3 groups, which become 3, 1, and 2 groups when sent. The first bundled data packet is sent with a delay of 12ms, and the second is sent with a delay of 35ms, which destroys the traffic timing characteristics.

[0079] An intelligent management output module is used to manage the transmission of interactive response information based on the generated interactive transmission information.

[0080] Example 3

[0081] As the third embodiment of the present invention, the focus is on combining the implementation processes of the first and second embodiments.

[0082] Some of the data in the above formulas are calculated based on their numerical values ​​and are not substituted into parameter units for calculation. At the same time, the contents not described in detail in this specification belong to the existing technology known to those skilled in the art.

[0083] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A virtual digital outbound intelligent customer service system based on generative AI, characterized by: include: The speech conversion and recognition module is used to convert, recognize and analyze the interactive information transmitted by the interactive information collection module to obtain text semantics, and determine the joint confidence by calculating the confidence of the text semantics and semantic information. At the same time, it matches the threshold interval to generate interactive response information and transmits it to the intelligent customer service comprehensive judgment module and the interactive transmission analysis module; The intelligent customer service comprehensive judgment module is used to combine interactive response information and historical interaction records to conduct multi-dimensional indicator monitoring and analysis. It calculates the knowledge-free anchor answer rate, the proportion of semantically ambiguous answers, and the correlation rate of high-frequency complaint issues. It then uses these three factors to judge the overall status of the intelligent customer service, generates normal or abnormal status information, and transmits it to the intelligent management output module. The interactive transmission analysis module is used to perform transmission encryption analysis on the obtained interactive response information, split the interactive response information into independent data packets, obtain data packet characteristics, and generate random byte segments based on the byte tail values ​​of the data packets, and combine them to obtain a combined data packet; Then the combined data packets are bundled and grouped and transmitted in a disorderly order and random delay manner to generate interactive transmission information and transmit it to the intelligent management output module.

2. The generative AI-based virtual digital outbound intelligent customer service system according to claim 1 is characterized in that: It also includes an interactive information acquisition module and an intelligent management output module; The interaction information acquisition module is used to obtain the interaction information, which is voice information, and transmit it to the voice conversion and recognition module; The intelligent management output module is used to display normal or abnormal status information and interactive transmission information to the corresponding management personnel.

3. The generative AI-based virtual digital outbound intelligent customer service system according to claim 1 is characterized in that: The specific method for the speech conversion recognition module to obtain text semantics is: Acquire interactive information and convert it into text information through speech recognition technology. Simultaneously, use natural language processing technology to identify the obtained text information to obtain text semantics. Then, use text similarity algorithm to remove duplicate corpus in the text semantics and generate preprocessed semantics. Obtain preprocessed semantics, combine them with non-voice data from customer service scenarios, and align the preprocessed semantics with business data through a multimodal model. At the same time, calculate the confidence of text semantics and voice information separately.

4. The generative AI-based virtual digital outbound intelligent customer service system according to claim 3 is characterized in that: The specific manner in which the speech conversion recognition module calculates the text semantic confidence and the speech information confidence is as follows: Calculate the text semantic confidence, process the audio by ASR model, obtain the output character probability of each frame, and then calculate the joint product of all character probabilities. Calculate the text semantic confidence ASR, where N is the number of frames, specifically i = 1, 2, ..., N, P i，j is the probability of recognizing character j in the i-th frame, and the average of the maximum probability of each frame is taken; Calculate the confidence of speech information, build the "acoustic feature-business semantics" matching model, extract the correlation between speech prosodic features and semantic intent, and use the formula The matching degree is calculated, and then the confidence level corresponding to the voice information is calculated according to the formula confidence level = matching degree × weight.

5. The virtual digital outbound intelligent customer service system based on generative AI according to claim 1 is characterized in that: The specific method for the speech conversion recognition module to generate interactive response information is: According to formula C z =α×ASR+(1-α)×P to calculate the joint confidence C z , where α is the text semantic confidence weight, P represents the voice information confidence, and the obtained joint confidence C z Match the corresponding threshold interval, determine the corresponding response strategy and generate interactive response information. The specific matching method is as follows: If the joint confidence C z ≥0.85, the corresponding response strategy is to directly output the answer; If 0.6≤joint confidence C z <0.85, then the corresponding response strategy is to trigger further questions for clarification; If the joint confidence C z <0.6, the corresponding response strategy is to transfer to manual or refuse to answer.

6. The virtual digital outbound intelligent customer service system based on generative AI according to claim 1 is characterized in that: The specific method for the intelligent customer service comprehensive judgment module to calculate the knowledge-free anchor answer rate, the semantically ambiguous answer ratio, and the high-frequency complaint question correlation rate is as follows: Calculate the rate of no-knowledge anchor answer, set the time period T, obtain all answer texts within the period, and use the semantic similarity algorithm to match the standard answer of the knowledge base to screen out answers with similarity less than the threshold value as no-knowledge anchor answer. According to the formula Calculate the knowledge-free anchor answer rate and compare it with the preset anchor answer rate. If it is lower than the preset value, it is normal; otherwise, it is abnormal. Calculate the proportion of semantically ambiguous answers. Use the ambiguity detection model, input the answer text, output the ambiguity probability, and if the ambiguity probability is ≥ 0.5, it is determined to be ambiguous. Count the number of ambiguous answers and calculate the answer according to the formula Calculate the proportion of semantically ambiguous answers and compare it with the preset value of ambiguous answer proportion. If it is lower than the preset value, it is normal; otherwise, it is abnormal. Calculate the correlation rate of high-frequency complaint issues. Based on historical complaint data, count the top K issues with the highest number of complaints to build a keyword library. The value of K is set by the operator. Perform keyword matching + semantic vector similarity detection on the consultation text to identify related issues. According to the formula The correlation rate of high-frequency complaint issues is calculated and compared with the preset correlation rate value. If it is higher than the preset value, it is abnormal, otherwise it is normal.

7. The generative AI-based virtual digital outbound intelligent customer service system according to claim 1 is characterized in that: The specific method for the intelligent customer service comprehensive judgment module to generate normal or abnormal status information is as follows: A judgment is made based on the analysis results of the answer rate without knowledge anchor points, the proportion of semantically ambiguous answers, and the correlation rate of high-frequency complaint issues. If any group of anomalies exists, it means that the overall status of the intelligent customer service is abnormal, and status adjustment information is generated. Conversely, if all are normal, it means that the overall status of the intelligent customer service is normal, and normal status information is generated.

8. The generative AI-based virtual digital outbound intelligent customer service system according to claim 1 is characterized in that: The specific method for the interactive transmission analysis module to obtain the combined data packet is: Obtain interactive response information, split it into independent data packets and label them as n, and n=1, 2, ..., m, where m represents the number of data packets. At the same time, obtain its corresponding data packet characteristics, obtain the byte size of the data packet, and obtain its byte tail value. At the same time, randomly generate byte segments with double the byte tail value as the standard byte number, and add the generated byte segments to the data packet characteristics to obtain a combined data packet.

9. The generative AI-based virtual digital outbound intelligent customer service system according to claim 1 is characterized in that: The specific method for the interactive transmission analysis module to generate interactive transmission information is as follows: All combined data packets are obtained and h groups are randomly selected to be bundled to obtain a bundled data packet. All bundled data packets are obtained by analogy and sent out of order. At the same time, random delay is used for periodic interval transmission and interactive transmission information is generated.

Images