Call center system and its multi-language intelligent voice translation method

Abstract

The application provides a call center system and a multilingual intelligent voice translation method thereof, and belongs to the technical field of artificial intelligence and natural language processing. The method comprises the following steps: performing multilingual interaction scene division on the call center system to generate scene classification data; constructing a stream processing pipeline architecture; dynamically dividing continuous voice into microsecond-level segments, decoding an ASR module in real time through an edge computing node to generate initial text data; matching a context memory library, extracting a historical semantic vector of the previous 5 rounds, and jointly encoding to generate context-enhanced text. Through the stream processing pipeline, context-enhanced translation, and seat correction instant learning technologies, a significant technical breakthrough and benefit improvement are realized; the end-to-end delay is less than or equal to 1.5 s (P95), the delay problem in the traditional serial processing mode is greatly reduced, and the strict requirements of real-time interaction are met.

Description

TECHNICAL FIELD

[0001] The application provides a call center system and a multilingual intelligent voice translation method thereof, and belongs to the technical field of artificial intelligence and natural language processing. BACKGROUND

[0002] Under the background of globalization, the daily call volume of call centers has increased dramatically, covering a large number of countries and languages. Traditional multilingual solutions rely on "manual translation + bilingual agents" or "segmented machine translation", which have significant defects.

[0003] On the one hand, the translation engine only processes the current sentence, ignoring the conversation history, resulting in pronoun reference errors and inconsistent professional terms. On the other hand, the agent's correction of the translated text cannot be fed back to the model in real time, and the errors continue to accumulate, making it difficult for the model to evolve. In addition, the serial processing of ASR, MT, and TTS results in an end-to-end delay often exceeding 2 seconds, which cannot meet the real-time interaction requirements.

[0004] The accuracy of general translation models drops sharply in professional fields such as finance and insurance, failing to meet the precision requirement of 90% in the field.

[0005] Existing technologies are generally described as "supporting multiple languages" and "real-time translation", without integrating agent feedback loops and context memory depth. The essence is still a "faster translator". Therefore, there is an urgent need for a high-precision, low-latency, and evolving multilingual intelligent voice translation method to break down language barriers and improve customer service quality. SUMMARY

[0006] The application provides a call center system and a multilingual intelligent voice translation method thereof to solve the problems mentioned in the background art:

[0007] The application provides a multilingual intelligent voice translation method of a call center system, which comprises:

[0008] S1, dividing the call center system into multiple language interaction scenarios, generating scenario classification data, and constructing a stream processing pipeline architecture;

[0009] S2, dynamically dividing continuous speech into microsecond-level segments, decoding the ASR module in real time through edge computing nodes to generate initial text data, matching the context memory library, extracting the first 5 rounds of historical semantic vectors, and jointly encoding to generate context-enhanced text;

[0010] S3, starting the field-adaptive translation engine, calling the professional corpus tuning parameters, dynamically calling the terminology library and anaphora resolution model, generating intermediate translated text containing terminology verification and pronoun correction, and pushing it to the agent end and activating the instant correction channel;

[0011] S4, correct data of the capture seat, align analysis error type and correction mode, generate structured error log; input model self-evolution module, adopt online learning algorithm to incrementally update parameters, and update context semantic vector synchronously;

[0012] S5, jointly optimize ASR, MT, TTS module delay, control end-to-end delay≤1.5s through dynamic load balancing and cache strategy; generate evaluation report, push visual dashboard and trigger risk warning.

[0013] The application provides a call center system, and the system comprises one or more processors and a memory for storing one or more programs.

[0014] The application has the following advantages: through the flow processing pipeline, context enhanced translation and seat correction instant learning, significant technical breakthrough and benefit improvement are realized; end-to-end delay≤1.5s (P95), the delay problem in the traditional serial processing mode is greatly reduced, the strict requirement of real-time interaction is met, and the customer experience is improved; BLEU score≥85%, and the field adaptation accuracy is≥90%, the problem that the accuracy of a general translation model suddenly decreases in a professional field is effectively solved, and the consistency and accuracy of terms in the financial and insurance industries are ensured; the 24-hour model self-evolution mechanism realizes continuous optimization of the translation model through instant backflow of seat correction, error accumulation is avoided, and the long-term stability and reliability of the system are improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The method described in the application is shown in the figure;

[0016] Figure 2 The flow processing pipeline data flow diagram of the application is shown in the figure;

[0017] Figure 3 The context enhancement processing mechanism schematic diagram of the application is shown in the figure;

[0018] Figure 4 The model self-evolution and performance optimization sequence diagram of the application is shown in the figure. DETAILED DESCRIPTION

[0019] The preferred embodiments of the application are described below with reference to the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the application, and are not used to limit the application.

[0020] One embodiment of the application is shown in the figure, Figure 1As shown, a multilingual intelligent voice translation method of a call center system, the method comprises:

[0021] S1, the call center system is divided into a multilingual interactive scene, and scene classification data containing different languages, business types and dialogue round characteristics are generated; a streaming processing pipeline architecture is constructed based on the scene classification data, such as Figure 2 As shown, the architecture integrates adaptive speech recognition (ASR), low-delay speech synthesis (TTS), and translation (MT) modules, and preset data flow trigger thresholds and parallel processing rules between the modules; Figure 2

[0022] S2, according to the streaming processing pipeline architecture, dynamic voice stream slicing is implemented, continuous speech is cut into microsecond-level segments according to semantic integrity, and real-time decoding of the ASR module is realized through an edge computing node to generate initial text data containing timestamps, language labels and confidence scores; the initial text data is matched with a context memory bank, the historical semantic vectors of the previous 5 rounds of dialogue are extracted, and joint coding is performed with the current segment to generate context-enhanced text data;

[0023] S3, based on the context-enhanced text data, a domain-adaptive translation engine is started, which performs initial parameter tuning through pre-trained financial, insurance and other professional domain corpora, and dynamically calls a terminology library and a coreference resolution model during translation to generate intermediate translation data containing professional term consistency verification and pronoun coreference correction; the intermediate translation data is pushed to the agent end interface and the instant correction feedback channel is activated, allowing the agent to mark and correct the incorrect translation through shortcut keys;

[0024] S4, the instant correction feedback channel is used to capture the agent's correction data in real time, align and analyze the corrected translation and the original incorrect translation, extract error types (such as term errors, syntax errors, and reference errors) and correction modes, and generate structured error logs; the structured error logs are input into the model self-evolution module, which uses an online learning algorithm to incrementally update the translation engine parameters and synchronously update the semantic vector representation in the context memory bank, achieving 24-hour continuous model optimization;

[0025] ​S5, according to the updated translation engine and context memory, jointly delay optimization is carried out on the ASR, MT, TTS module in the streaming pipeline, through dynamic load balancing and cache prefetch strategy, the end-to-end processing delay is controlled in the range of <=1.5s (P95); based on the optimized system, a comprehensive evaluation report containing multilingual translation accuracy (BLEU >=85%), real-time index and model evolution state is generated, and the report is pushed to the operation and maintenance end in real time through a visual dashboard, and a risk warning mechanism is triggered, when it is detected that the translation accuracy is continuously lower than the threshold or the delay is over-standard, fault diagnosis and repair suggestions are automatically generated.

[0026] The working principle and effect of the above technical solution are:

[0027] Through multilingual real-time translation + scene classification adaptation, 80+ countries / regions customers can be supported by single language seat service, the dependence on small language seat recruitment is reduced, the inefficiency problem of traditional manual translation intervention is avoided, and the global business service coverage capability is improved.

[0028] Through the streaming processing pipeline + edge computing + cache prefetch strategy, the end-to-end translation delay is controlled in <=1.5s (P95), the customer waiting time is reduced, the communication gap caused by translation lag is avoided, and the service response efficiency and one-time solution rate are enhanced.

[0029] Through the field self-adaptive translation engine + context memory + seat correction backflow optimization, the translation accuracy of BLEU >=85% is realized, the problems such as term error and reference ambiguity are reduced, the repeated communication cost is avoided, and the translation accuracy and professional scene adaptation degree are improved.

[0030] Through full-link delay monitoring + dynamic load balancing + intelligent warning mechanism, the system bottleneck is quickly located, the fault omission or delayed processing situation is reduced, the stability risk under high concurrency of the system is avoided, and the operation and maintenance efficiency and system reliability are enhanced.

[0031] Through data transmission encryption (TLS1.3+AES-256) + recording desensitization processing, the compliance requirements of GDPR / Equal Protection 3.0 and the like are met, the information leakage risk is reduced, the compliance penalty risk is avoided, and the security of data transmission and storage is improved.

[0032] Through 24-hour self-evolution of the model + modular architecture design, the newly added languages and business scenes are continuously adapted, the complexity of system iteration and upgrading is reduced, the compatibility problem caused by function expansion is avoided, and the flexibility and scalability of the system are enhanced.

[0033] An embodiment of the application, the S1, comprises:

[0034] S11, by analyzing the call center historical call recording, work order data and business account, 30+ target language samples of 80+ countries / regions are collected, the dialogue scene characteristics of different business types (finance, insurance, logistics, government and enterprise) are extracted, the dialogue scene characteristics include consultation frequency, core appeal, commonly used terms and dialogue round distribution, and a scene original feature set is generated;

[0035] S12, based on the scene original feature set, an unsupervised clustering algorithm is used to classify the scene, three categories of high-frequency basic scene, professional field scene and special interaction scene are divided, a language label, a business label and a round label are assigned to each scene, and a structured scene classification data is generated;

[0036] S13, taking the scene classification data as the adaptation basis, a full-link flow architecture of access, processing and output is constructed, an adaptive speech recognition (ASR) module, a context enhanced translation (MT) module and a low-delay speech synthesis (TTS) module are integrated, the function boundary and data flow path of each module are determined, and a modular architecture blueprint is formed;

[0037] S14, preset the data flow trigger condition between each module, define the trigger delay threshold after ASR decoding, the context calling trigger rule of MT module and the synthesis starting threshold of TTS, formulate parallel processing priority strategy (high-frequency scene priority, short dialogue priority), and ensure the efficient cooperation of pipeline.

[0038] The working principle and effect of the above technical scheme are:

[0039] By collecting multilingual samples and full-scene features, covering 80+ countries / regions and core appeals of multiple business types, the comprehensiveness of scene features is improved, and the omission of key interaction features is reduced;

[0040] By using an unsupervised clustering algorithm to classify scenes and assign exclusive labels, the accuracy of scene classification is improved, and the adaptation deviation caused by confusion of different types of scenes is avoided;

[0041] Based on the scene classification data, a modular flow architecture is constructed, the function boundary and data path of each module are clear, the adaptability of the system architecture is enhanced, and the conflict risk during module integration is reduced;

[0042] Presetting the module trigger condition and the parallel priority strategy makes high-frequency and short dialogue scenes be processed first, improves the pipeline cooperation efficiency, reduces the redundant time consumption of data waiting between modules, and lays a foundation for efficient architecture for subsequent real-time translation.

[0043] An embodiment of the application, the S13, comprises:

[0044] With the language label, business label and round label in the scene classification data as the core basis, the speech processing flow requirements of different scenes (high-frequency basic scene, professional field scene, special interaction scene) are disassembled to determine the speech format (such as RTP / MP3) supported by the access layer, the core capabilities (such as multi-language recognition, field translation) covered by the processing layer, and the terminal (agent interface, customer call end) connected by the output layer, and a list of architecture design requirements is generated;

[0045] Based on the list of architecture design requirements, the three levels of the whole link architecture are divided: the access layer (responsible for speech signal reception and format standardization), the processing layer (carrying core speech translation operation), and the output layer (responsible for translation result pushing and display), and the hardware resource allocation rules of each level are defined (such as deploying edge nodes in the access layer, using distributed computing in the processing layer), and a framework diagram of the architecture level is generated;

[0046] The adaptive speech recognition (ASR) module, the context enhanced translation (MT) module, and the low delay speech synthesis (TTS) module are respectively deployed to the corresponding nodes of the processing layer, and the core function boundaries of each module are determined: the ASR module is responsible for speech-text conversion (including noise suppression and hot word recognition), the MT module is responsible for text-multilingual translation conversion (including context memory and domain terminology adaptation), and the TTS module is responsible for translation-speech conversion (including brand voice cloning and emotion adaptation), and a module function specification is formed;

[0047] According to the scene processing logic and module function, the data flow direction between modules (such as access layer-ASR-MT-TTS-output layer) is designed, the data transmission format (such as ASR output timestamped JSON text, TTS output 16kHz / 16bit PCM audio stream) and interaction protocol (such as Websocket real-time transmission) are determined, and a data flow path diagram is drawn;

[0048] The architecture level framework diagram, module function specification and data flow path diagram are summarized, the performance parameters of each module (such as ASR word error rate ≤8%, MT delay ≤300ms) and the interface specifications between levels are supplemented, and a complete modular architecture blueprint is formed.

[0049] The working principle and effect of the above technical solution are:

[0050] By disassembling the speech processing requirements of different scenes and generating a list, the architecture design accurately matches the business and language requirements, improving the fit of the requirements, and reducing the problem of architecture design and actual scene disconnection;

[0051] The three-level architecture of access, processing and output is divided and the resource allocation rules are clearly defined, enhancing the logicality of the architecture level, reducing the waste caused by hardware resource mismatch, and fully utilizing the advantages of edge nodes and distributed computing;

[0052] The core function boundary of the ASR / MT / TTS module is clear, which avoids function overlap or omission between modules, improves the smoothness of module cooperation, and reduces function conflicts during integration;

[0053] The data flow direction, transmission format and interaction protocol are specified, which ensures real-time and efficient data flow, enhances the continuity of the whole translation link, and lays a foundation for subsequent low-delay processing;

[0054] The complete blueprint containing performance parameters and interface specifications is formed, which improves the feasibility of the architecture, reduces the complexity of subsequent development, integration and iteration, and provides a clear basis for system building.

[0055] In an embodiment of the present application, the S14 comprises:

[0056] Based on the data flow path of each module in the formed modular architecture blueprint, combined with the generated scene classification data (label attributes of high-frequency basic scene, professional field scene and special interaction scene), the trigger scene demand of ASR to MT, MT to TTS and cross-module cooperation is sorted out, the core demand of data flow trigger in different scenes is determined (such as low-delay trigger for high-frequency scene, accurate trigger for professional scene), and a module trigger demand list is generated;

[0057] Based on the low-delay core demand in the trigger demand list, combined with the performance index of ASR module delay ≤400ms, the trigger delay threshold of ASR decoding completion to push data to MT module is set (such as ≤100ms, reserving MT processing buffer time), and the threshold adjustment mechanism is synchronized (such as dynamically relaxed to ≤150ms when peak concurrency), and an ASR trigger threshold configuration document is generated;

[0058] According to the accuracy demand in the trigger demand list and the round label of scene classification data, the context calling trigger condition of MT module is set: when the dialogue round is ≥2 rounds, the context memory library is automatically called; when the text contains professional field label (such as finance), the field term library is triggered to call; at the same time, the trigger failure scene (such as resetting the context when the dialogue interruption is more than 30s) is defined, and the MT context calling rule table is formed;

[0059] Combined with the trigger demand list and the requirements of TTS module delay ≤200ms and MOS ≥4.2, the TTS synthesis start threshold is set: when the translation text integrity of MT module output is ≥95% (no untranslated term / syntax fault), and the scene matching degree is ≥90% (the translation text is consistent with the scene business label), the TTS synthesis is started; the threshold exemption mechanism is set for special interaction scene (such as emergency fault declaration), and the TTS start threshold specification is generated;

[0060] Based on the frequency and dialogue length attributes of the scene classification data, the scenes are prioritized (high-frequency basic scenes > professional field scenes > special interaction scenes), and then the scenes with the same priority are divided according to the dialogue length (short dialogue ≤ 3 sentences is preferred to long dialogue), the resource allocation proportion of different priority scenes is determined (for example, high-frequency scenes account for 60% of the computing power, professional scenes account for 30%), and a parallel processing priority strategy matrix is formed;

[0061] The ASR trigger threshold, MT context rules, TTS start threshold and parallel strategy matrix are integrated, the verification mechanism (such as automatic alarm when the threshold is exceeded) and the update cycle (monthly iteration based on scene data) of each configuration item are supplemented, and a full-link data flow trigger and priority configuration scheme is generated.

[0062] The working principle and effect of the above technical solution are:

[0063] By sorting out the trigger core demands of different scenes and generating a list, the module configuration accurately matches the scene demand, improves the pertinence of the trigger rules, and reduces the resource waste caused by blind configuration;

[0064] Setting the ASR trigger threshold and dynamic adjustment mechanism not only reserves the MT processing buffer time, but also adapts to the peak concurrent scene, ensuring the timeliness of data pushing and reducing the waiting time between modules;

[0065] Defining the MT context calling conditions and invalidation rules, combining the round and professional label to accurately trigger the memory library and the terminology library, improves the coherence and professionalism of translation, and reduces the problems of ambiguous reference and terminology deviation;

[0066] Formulating the TTS synthesis threshold and special scene exemption mechanism, taking into account the translation quality and urgent demand, improving the adaptability of speech synthesis, and avoiding service delay caused by threshold rigidity;

[0067] According to the scene frequency and length, the priority is divided and the computing power is allocated, so that the high-frequency short dialogue is processed preferentially, the system resource utilization rate is improved, and the response delay of the core scene is reduced;

[0068] Integrating the configuration scheme and supplementing the verification mechanism and the update cycle, the stability and iterability of the configuration are enhanced, the complexity of subsequent operation and adjustment is reduced, and the efficient cooperation of the full link is ensured.

[0069] In an embodiment of the present application, the S2 comprises:

[0070] S21, receiving an accessed voice stream (16kHz / 16bit PCM format), the voice stream includes access through PSTN / WebRTC / SIP and the like, based on the semantic pause characteristics and the voice energy change, the continuous voice stream is cut into a microsecond-level semantic complete segment, cross semantic splitting is avoided, and fragmented voice data is generated;

[0071] S22, push the fragmented voice data to the edge computing node, start the streaming recognition function of the ASR module, combine the noise suppression algorithm and the business hot word library, and perform real-time decoding on the voice segment, output initial text data containing a timestamp, a language label (confidence ≥ 90%) and an identification confidence score, and ensure that the word error rate is ≤8%;

[0072] S23, based on the conversation identification in the initial text data, retrieve the context memory library, extract the text content of the first 5 rounds of conversation and convert it into a semantic vector, establish an associated mapping of the historical conversation and the current segment, and generate a set of historical semantic vectors;

[0073] S24, using the Transformer coding structure, jointly encode the initial text data of the current segment and the set of historical semantic vectors, fuse cross-round context information, eliminate ambiguity and semantic discontinuity, and generate context-enhanced text data.

[0074] The working principle and effect of the above technical solution are:

[0075] By cutting the voice stream according to semantic pauses and energy changes, the semantic integrity of the fragments is ensured, the basic quality of subsequent processing is improved, and the understanding deviation caused by cross-semantic splitting is reduced;

[0076] Relying on the edge computing node and the ASR streaming recognition, combining noise suppression and business hot word library, the accuracy of voice decoding is improved, the word error rate is controlled to be ≤8%, and the environmental noise and professional term recognition error are reduced;

[0077] Precise extraction of semantic vectors of the first 5 rounds of conversation and establishment of association enhance the relevance of historical and current conversations, and reduce the problem of cross-round semantic disconnection;

[0078] By fusing context information through the Transformer coding structure, ambiguity and semantic discontinuity are eliminated, the coherence and usability of the text data are improved, high-quality input is provided for subsequent field translation, and the processing real-time performance is ensured, which meets the low-latency requirement of the system.

[0079] In one embodiment of the application, the S22 comprises:

[0080] According to the business scene label (such as finance / insurance) corresponding to the fragmented voice data, it is pushed to the nearest edge computing node (matching the scene corresponding to the computing resource pool), and the node receives the data format (16kHz / 16bitPCM) and integrity (no fragment loss) after verification, and generates a data reception confirmation signal;

[0081] Based on the data reception confirmation signal, the ASR module streaming recognition function of the edge node is triggered, and a self-adaptive noise suppression algorithm (for common background noise and current sound in call centers) is loaded to perform real-time noise reduction processing on the fragmented voice data, generating denoised voice segments.

[0082] According to the service label of the fragmented voice data (such as the financial scenario associated with the financial interest and account balance hot words, and the insurance scenario associated with the policy claim and premium payment hot words), the corresponding business hot word library is called, the hot word weight is injected into the ASR recognition model, the recognition accuracy of professional terms is improved, and the voice recognition intermediate result with hot word labels is generated.

[0083] The voice recognition intermediate result with hot word labels is decoded in real time, the text content is extracted, and the collection time of the fragmented voice data is automatically associated to generate a time stamp, the language label is output through a language detection model (and the confidence is calculated to ensure ≥90%), the recognition confidence score is generated based on the recognition matching degree, and is packaged as structured data to preliminarily form an initial text data prototype.

[0084] The character-level comparison algorithm (comparing the initial text data prototype with the standard voice sample text) is used to check the word error rate. If the word error rate is ≤8%, the data is confirmed as valid initial text data. If the word error rate is >8%, the ASR model parameter fine-tuning (such as adjusting the hot word weight) is triggered and re-decoded until the word error rate requirement is met, and the standard initial text data is finally output.

[0085] The working principle and effect of the above technical solution are:

[0086] By pushing to the nearest edge node according to the business label and verifying the data format and integrity, the adaptability and reliability of data transmission are improved, and the format error and fragment loss problems are reduced.

[0087] Loading the self-adaptive noise suppression algorithm to process background noise, current sound and other interference reduces the influence of environmental noise on recognition and enhances the clarity of voice signals.

[0088] According to the scene, the corresponding business hot word library is called and the weight is injected, which improves the recognition accuracy of professional terms in the fields of finance, insurance and the like, and reduces the recognition deviation of domain words.

[0089] When decoding, the time stamp, language label (confidence ≥90%) and recognition confidence score are packaged, which enriches the data dimensions of the initial text and enhances the usability of the data.

[0090] By character-level comparison to check the word error rate and dynamically fine-tune the model, the output data word error rate is ensured to be ≤8%, which reduces the voice recognition error and provides high-quality input for subsequent context enhancement processing, as shown in Figure 3 .

[0091] In one embodiment of the present application, the S24 comprises:

[0092] The generated initial text data (containing time stamp, language label) and the generated historical semantic vector set are obtained, the initial text data is converted by word embedding (text characters are mapped to fixed dimension vectors), the dimension of the historical semantic vector set is adjusted to be consistent with the initial text word embedding vector, and a joint data set of the current text vector and the historical semantic vector with uniform format is generated;

[0093] Based on the business scene label (finance / insurance / logistics, etc.) corresponding to the joint data set, the Transformer encoding structure is initialized: the number of encoder layers (such as 6 layers for professional field scene and 4 layers for high-frequency basic scene) is set, the number of attention heads (8 attention heads, which strengthens the capture of cross-turn semantic association) is set, and the self-attention mechanism is optimized for the dialogue scene (the turn position coding is increased to distinguish the time sequence relationship between the current and historical dialogues), and the scene configured Transformer encoder is generated;

[0094] The aligned joint data set is input into the scene configured Transformer encoder, the association weight of the current text vector and the historical semantic vector is calculated through the self-attention layer (such as the weight of the pronoun-antecedent and the term-context definition is preferentially strengthened), and then the feature fusion is performed through the feedforward neural network, and the fusion encoding vector containing the cross-turn semantic association information is generated;

[0095] The fusion encoding vector is decoded (converted into a text segment), and through the semantic coherence detection algorithm (such as judging whether the reference is clear and the semantics is discontinuous based on the pre-trained dialogue model), the problems such as the pronoun without corresponding antecedent and the professional term without context explanation are identified and corrected, and the preliminary context enhanced text is generated;

[0096] According to the extension requirement that the ASR word error rate is less than or equal to 8%, the preliminary context enhanced text is subjected to semantic integrity verification (to ensure that there is no loss of key information) and logical consistency verification (to ensure that there is no conflict between cross-turn semantics), and if the verification is passed, the final context enhanced text data is determined; if not, the Transformer encoder is fine-tuned to adjust the attention weight and then re-encoded until the quality requirement is met.

[0097] The working principle and effect of the above technical solution are:

[0098] By unifying the format of the current text and the historical semantic vector, the compatibility of data encoding is improved, and the feature fusion error caused by inconsistent dimensions is reduced;

[0099] The number of Transformer encoder layers and attention mechanism are dynamically configured according to business scenarios, the adaptability of the model to different scenarios is enhanced, and the capture accuracy of cross-turn semantic association is improved.

[0100] The association weight of pronoun-antecedent and term-context is preferentially enhanced, the reference ambiguity and professional term understanding deviation are reduced, and the accuracy of semantic association is enhanced.

[0101] Through semantic coherence detection and problem correction, the occurrence rate of semantic discontinuity, pronoun without reference and other problems is reduced, and the coherence of the text is improved.

[0102] The model is dynamically fine-tuned through semantic integrity and logical consistency double verification, ensures that the output text quality meets the standard, reduces the loss of key information and cross-turn semantic conflict, and provides high-quality input for subsequent field translation.

[0103] One embodiment of the present application, the S3, comprises:

[0104] S31, for the professional field label in the scene classification data, calling the pre-trained field (finance, insurance, logistics, etc.) corpus, the initial parameter of MT module is optimized, the professional term translation weight is optimized, and the translation accuracy in the field (field adaptation degree is greater than or equal to 90%) is improved.

[0105] S32, in the translation process of MT module, the professional terms and pronouns in the context enhanced text data are detected in real time, the consistency verification is dynamically called by calling the industry term library, the reference resolution model is started to correct the reference ambiguity problem, and the preliminary translation data is generated.

[0106] S33, the quality of the preliminary translation is evaluated by BLEU score algorithm, whether the translation accuracy is greater than or equal to 85% is evaluated, the secondary optimization is carried out on the translation which does not meet the standard, and the intermediate translation data containing term consistency mark and reference correction instruction is generated.

[0107] S34, the intermediate translation data is synchronously pushed to the interface of the seat end, the original text and the translation are displayed on the same screen, the shortcut key type instant correction feedback channel is activated, the error translation is marked, modified and submitted by the seat, and the two-way interaction mechanism is established.

[0108] The working principle and effect of the above technical scheme are as follows:

[0109] By calling the professional field pre-training corpus to optimize the MT module parameters, the professional term translation weight is optimized, the translation adaptation degree (greater than or equal to 90%) in the fields of finance, insurance and the like is improved, and the cross-field term translation deviation is reduced.

[0110] Real-time detection of professional terms and pronouns in text, dynamic linkage of term library for consistency check, and activation of reference resolution model to enhance the accuracy and standardization of the translation, reduce the problem of ambiguous reference and chaotic terms;

[0111] Through the BLEU score algorithm evaluation and secondary optimization of the substandard translation, ensure that the translation accuracy is greater than or equal to 85%, reduce the error of the translation, improve the stability of the translation quality;

[0112] Realize the display of the original text and the translation on the same screen, activate the shortcut instant correction channel, facilitate the operator to quickly mark the modification error, improve the work efficiency of the operator, reduce the repeated communication cost caused by the translation error, and at the same time, establish a two-way interaction mechanism, accumulate effective data for subsequent model self-evolution.

[0113] One embodiment of the present application, the S4 comprises:

[0114] S41, the correction operation of the operator is captured through the instant correction feedback channel, including the error marking position, the corrected correct translation and the correction note, the original correction data collected is subjected to format standardization processing, the data structure and field definition are unified, and the standard correction data set is generated;

[0115] S42, the corrected translation in the standard correction data is aligned with the corresponding original error translation sentence by sentence, the natural language processing algorithm is used to identify the error type, the error type includes term error, syntax error, reference error and semantic deviation, the correction mode of various errors is extracted, and the error feature library is formed;

[0116] S43, based on the error feature library and the correction mode, a structured error log is generated, the structured error log includes error type, scene label, language information and correction example, and is stored in a log database in time dimension and scene dimension, and provides data support for model optimization;

[0117] S44, the structured error log is input into the model self-evolution module, as shown in Figure 4 The online learning algorithm is used to incrementally update the translation parameters of the MT module, and the semantic vector representation in the context memory library is updated synchronously, so that the correction experience is integrated into the model within 24 hours and takes effect.

[0118] The working principle and effect of the above technical scheme are:

[0119] By standardizing the format and field definition of the correction data of the operator, the availability of the correction data is improved, and the subsequent processing obstacles caused by data confusion are reduced;

[0120] The corrected translation before and after sentence alignment is revised, various error types are accurately identified and correction modes are extracted, the systematization of error analysis is enhanced, and the probability of repeated occurrence of similar translation errors is reduced.

[0121] The structured error log containing scene tags and language information is generated and stored in categories, the data support for model optimization is more targeted, and the interference of invalid data is reduced.

[0122] The translation parameter incremental update and semantic vector synchronous optimization are realized through an online learning algorithm, the correction experience is ensured to take effect within 24 hours, the translation accuracy is continuously improved, the workload of manual repeated correction is reduced, and the self-adaptive evolution ability of the model for different scenes and languages is enhanced.

[0123] In one embodiment of the present application, the S5 comprises:

[0124] S51, the processing delay of the ASR, MT and TTS modules is monitored in the whole link, the delay bottleneck node is located, the dynamic load balancing algorithm is used to distribute the processing task to the idle node, the data transmission path between the modules is optimized, and the redundant time consumption is reduced.

[0125] S52, based on the language distribution and scene frequency in the historical call data, a cache prefetch rule is formulated, the high-frequency language model, the popular scene term library and the commonly used semantic vector are loaded in advance, the module calling response time is shortened, and the end-to-end processing delay is controlled within the range of ≤1.5s (P95).

[0126] S53, the translation accuracy (BLEU value), real-time index (module delay, end-to-end delay), model evolution state (correction adoption rate, error rate reduction amplitude) and system stability data are collected, and a multi-dimensional comprehensive evaluation report is generated by using a weighted scoring method.

[0127] S54, the comprehensive evaluation report is pushed to the operation and maintenance end in real time through a visual dashboard, a translation accuracy threshold (lower than 80%) and a delay exceeding threshold (higher than 2s) are set, when an index anomaly is detected, a risk early warning mechanism is automatically triggered, and a fault diagnosis report and targeted repair suggestion are generated.

[0128] The working principle and effect of the above technical solution are:

[0129] Through the whole link monitoring of the module delay and the location of the bottleneck node, the dynamic load balancing is combined to distribute the task and optimize the data transmission path, the system resource utilization is improved, and the module congestion and redundant time consumption are reduced.

[0130] Based on the language distribution and scene frequency of the historical call, a cache prefetch rule is formulated, high-frequency resources are loaded in advance, the module calling response time is reduced, the end-to-end delay is controlled within ≤1.5s (P95), and the real-time performance of the translation service is enhanced.

[0131] Collecting translation accuracy, real-time, model evolution and stability multidimensional data to generate a comprehensive evaluation report, improves the control accuracy of the system state, and reduces the resource waste caused by blind optimization;

[0132] Through the visual dashboard to push the report, combined with the abnormal early warning mechanism and the targeted diagnosis suggestion, the accuracy and response efficiency of the operation and maintenance decision are enhanced, the time cost of fault troubleshooting and repair is reduced, the long-term stable operation of the system is ensured, and the translation quality and real-time performance are continuously maintained.

[0133] An embodiment of the present application, the S51, comprises:

[0134] According to the delay standard of each module in the efficiency requirement (ASR≤400ms, MT≤300ms, TTS≤200ms), the monitoring object is determined as the ASR module (decoding delay), the MT module (translation delay), the TTS module (synthesis delay) and the inter-module data transmission delay (such as ASR--MT, MT--TTS), the calculation method of each delay index (the time difference from the data entering the module to the output result) is defined, and the full-link delay monitoring index list is generated;

[0135] Based on the Prometheus+Grafana monitoring scheme in the operation and maintenance environment requirement, delay data collection probes are deployed in the ASR, MT, TTS modules and data transmission nodes, real-time capture of each module processing delay and transmission time data, storage in the monitoring database at a frequency of seconds, and real-time display of the delay change trend through the Grafana dashboard, to generate a real-time delay data set;

[0136] The real-time delay data set is compared and analyzed, the actual delay of each module is compared with the preset threshold (ASR≤400ms, MT≤300ms, TTS≤200ms), and the modules exceeding the threshold are selected; at the same time, the transmission time of the modules is analyzed, the redundant nodes with time consumption exceeding 50ms in the transmission path are identified, the information of the modules and redundant nodes exceeding the threshold is summarized, and a bottleneck node analysis report is generated;

[0137] According to the high-load module / node information in the bottleneck node analysis report, an adaptive dynamic load balancing algorithm (such as weighted round robin algorithm, higher weight for idle nodes) is used to set the task allocation rules: the processing tasks (such as ASR decoding tasks) of the high-load modules are preferentially allocated to the idle nodes with CPU occupancy rate <30%, and the single-node task concurrency is limited (not more than 80% of the maximum carrying capacity), to generate a dynamic task allocation scheme;

[0138] In combination with the redundant transmission node information in the bottleneck node analysis report, the intermediate nodes without data processing function in the transmission path are deleted, the Websocket protocol in the open interface is used to replace the traditional HTTP protocol to transmit data (reduce the time consumption of connection establishment), the data flow path between modules is re-planned, and the optimized transmission path diagram is generated.

[0139] The whole link delay data after optimization (dynamic load balancing+path optimization) is collected, and the delay changes before and after optimization are compared (such as whether the ASR delay is reduced to 400ms or less, and whether the transmission time is reduced), if there is still a threshold delay, the bottleneck is re-analyzed and the load balancing strategy or transmission path is adjusted until the whole link delay meets the threshold requirements of each module, and an optimization effect verification report is generated.

[0140] The working principle and effect of the above technical solution are:

[0141] By specifying the delay monitoring objects and calculation methods of each module, a standard index list is generated, which improves the accuracy of delay monitoring and reduces the misjudgment of bottlenecks caused by ambiguous indicators.

[0142] Relying on the deployment of Prometheus+Grafana collection probes, delay data is stored and displayed in real time at a level of seconds, which enhances the traceability of delay data and reduces the analysis obstacles caused by data loss.

[0143] By comparing the threshold of the modules and redundant nodes, a targeted bottleneck report is generated, which improves the efficiency of bottleneck positioning and reduces the invalid investment of blind optimization.

[0144] Adaptive dynamic load balancing algorithm is used to allocate tasks, idle nodes are preferentially scheduled and the concurrency is limited, which improves the resource utilization and reduces the risk of delay soaring caused by module congestion.

[0145] The redundant transmission nodes are deleted and the Websocket protocol is used to optimize the data flow path, which reduces the connection establishment and transmission time consumption and enhances the data interaction efficiency between modules.

[0146] Through optimization effect verification and iterative adjustment, it is ensured that the whole link delay meets the threshold requirements of each module, which improves the effectiveness of delay optimization and lays a solid foundation for end-to-end delay control in ≤1.5s (P95).

[0147] In one embodiment of the present application, a call center system, the system comprises: one or more processors; a memory for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multilingual intelligent voice translation method of the call center system according to any one of the above.

[0148] Obviously, many modifications and variations of the present application are possible in light of the above teachings. It is, therefore, to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A multilingual intelligent speech translation method for a call center system, characterized in that, The method comprises: S1, multi-language interaction scene division is performed on the call center system to generate scene classification data; and a stream processing pipeline architecture is constructed; S2, continuously spoken language is dynamically fragmented into microsecond-level segments, an initial text data is generated by real-time decoding an adaptive speech recognition module through an edge computing node, a context memory library is matched, a history semantic vector set of the previous 5 rounds is extracted, and the initial text information of the current segment is jointly encoded with the history semantic vector set to generate context-enhanced text; S3, a domain adaptive translation engine is started based on the context-enhanced text data, the engine performs initial parameter optimization through a pre-trained professional domain corpus, and dynamically calls a terminology library and a coreference resolution model during translation to generate intermediate translation data containing professional term consistency verification and pronoun coreference correction; the intermediate translation data is pushed to a seat end interface, and an instant correction feedback channel is activated to allow the seat to mark and correct the error translation through a shortcut key; S4, by capturing the seat correction data, the corrected translation and the original error translation are aligned and analyzed to extract error types and correction modes, and generate structured error logs; input a model self-evolution module, which updates the translation engine parameters incrementally using an online learning algorithm, and synchronously updates the semantic vector representation in the context memory library; S5, jointly optimize the ASR, MT and TTS module delays, the ASR is an adaptive speech recognition module, the MT is a translation module, and the TTS is a low-delay speech synthesis module; through dynamic load balancing and cache strategy, the end-to-end delay is controlled to be less than or equal to 1.5s; an evaluation report is generated, a visual dashboard is pushed, and a risk early warning mechanism is triggered.

2. The multi-lingual intelligent speech translation method for a call center system according to claim 1, wherein, The S1 comprises: S11, by analyzing call center historical call recording, work order data and business account, 30+ target language samples of 80+ countries / regions are collected, dialog scene features of different business types are extracted, and a scene original feature set is generated; S12, based on the scene original feature set, an unsupervised clustering algorithm is used to classify the scenes in layers, three categories of high-frequency basic scenes, professional domain scenes and special interaction scenes are divided, language labels, business labels and round labels are assigned to each category of scenes, and structured scene classification data is generated; S13, based on the scene classification data, a full-link stream architecture is constructed for access, processing and output, an adaptive speech recognition module, a context-enhanced translation module and a low-delay speech synthesis module are integrated, the function boundaries and data flow paths of each module are determined, and a modular architecture blueprint is formed; S14, preset the data flow trigger conditions between the modules, define the trigger delay threshold after ASR decoding, the context calling trigger rule of the MT module and the TTS synthesis start threshold, and develop a parallel processing priority strategy.

3. The method of claim 2, wherein the call center system is a multi-lingual intelligent voice translation system. The S13 comprises: Based on the language labels, business labels and round labels in the scene classification data as the core basis, the speech processing process requirements of different scenes are disassembled, the speech formats supported by the access layer, the core capabilities covered by the processing layer and the terminals connected by the output layer are determined, and a list of architecture design requirements is generated; Based on the architecture design requirement list, divide the three levels of the whole link architecture: access layer, processing layer, output layer, define the hardware resource allocation rules of each level, and generate the architecture level framework diagram; Deploy the adaptive speech recognition module, context enhanced translation module and low delay speech synthesis module to the corresponding nodes of the processing layer respectively, determine the core function boundary of each module, and form the module function specification; According to the scene processing logic and module function, design the data flow direction between modules, determine the data transmission format and interaction protocol, and draw the data flow path diagram; Summarize the architecture level framework diagram, module function specification and data flow path diagram, supplement the performance parameters of each module and the interface specification between levels, and form a complete modular architecture blueprint.

4. The method of claim 1, wherein the call center system is a multi-lingual intelligent voice translation system. The S2 comprises: S21, receiving the accessed voice stream, cutting the continuous voice stream into microsecond-level semantic complete fragments based on the semantic pause feature and the voice energy change, and generating fragmented voice data; S22, pushing the fragmented voice data to the edge computing node, starting the stream recognition function of the ASR module, combining the noise suppression algorithm and the business hot word library to decode the voice fragments in real time, and outputting the initial text data; S23, based on the conversation identifier in the initial text data, retrieving the context memory library, extracting the text content of the previous 5 rounds of conversation and converting it into a semantic vector, establishing an association mapping between the historical conversation and the current fragment, and generating a set of historical semantic vectors; S24, using the Transformer coding structure to jointly encode the initial text data of the current fragment and the set of historical semantic vectors, fuse the cross-round context information, and generate context enhanced text data.

5. The multi-lingual intelligent speech translation method for a call center system according to claim 4, wherein, The S22 comprises: According to the business scenario label corresponding to the fragmented voice data, it is pushed to the nearest edge computing node, and the node verifies the data format and integrity after receiving, and generates a data reception confirmation signal; Based on the data reception confirmation signal, trigger the stream recognition function of the ASR module of the edge node, and load the adaptive noise suppression algorithm at the same time, to process the fragmented voice data in real time, and generate denoised voice fragments; According to the business label of the fragmented voice data, call the corresponding business hot word library, inject the hot word weight into the ASR recognition model, and generate the voice recognition intermediate result with hot word label; The voice recognition intermediate result with hot word label is decoded in real time, the text content is extracted, the collection time of the fragmented voice data is automatically associated to generate a timestamp, the language label is output through the language detection model, and the recognition confidence score is generated based on the recognition matching degree, and is packaged as structured data, to form the initial text data prototype; Use the character-level comparison algorithm to check the word error rate, if the word error rate is less than or equal to 8%, the data is confirmed as valid initial text data; if the word error rate is greater than 8%, trigger the ASR model parameter fine tuning and re-decode until the word error rate requirement is met, and finally output the initial text data.

6. The multilingual intelligent speech translation method for the call center system according to claim 1, characterized in that, The S3 comprises: S31, for the professional field label in the scene classification data, call the pre-trained domain corpus to optimize the initial parameters of the MT module; S32, in the MT module translation process, real-time detection of context enhanced text data of professional terms and pronouns, dynamic call industry terminology library for consistency check, generate preliminary translation data; S33, through the BLEU score algorithm for quality evaluation of preliminary translation, evaluate translation accuracy whether ≥ 85%, the translation of the unqualified for the second optimization, generate intermediate translation data; S34, the intermediate translation data is pushed to the interface of the seat, the original text and translation are displayed on the screen, the shortcut key type instant correction feedback channel is activated, the seat can mark, modify and submit the wrong translation, and a two-way interaction mechanism is established.

7. The method of claim 1, wherein the call center system is a multi-lingual intelligent voice translation system. The S4 comprises: S41, capture the correction operation of the seat through the instant correction feedback channel, standardize the format of the collected original correction data, and generate a standard correction data set; S42, align the corrected translation in the standard correction data with the corresponding original error translation sentence by sentence, identify the error type using natural language processing algorithm, extract the correction mode of various errors, and form an error feature library; S43, based on the error feature library and the correction mode, generate a structured error log, and store it in the log database according to time dimension and scene dimension; S44, input the structured error log into the model self-evolution module, and use online learning algorithm to incrementally update the translation parameters of the MT module, and update the semantic vector representation in the context memory library.

8. The method of claim 1, wherein the call center system is a multi-lingual intelligent voice translation system. The S5 comprises: S51, monitor the processing delay of ASR, MT and TTS modules in the whole link, locate the delay bottleneck node, and distribute the processing task to the idle node by using dynamic load balancing algorithm; S52, based on the language distribution and scene frequency in the historical call data, make cache prefetching rules, load high-frequency language models, popular scene terminology library and common semantic vectors in advance, and shorten the module calling response time; S53, collect translation accuracy, real-time index, model evolution state and system stability data, and generate a multi-dimensional comprehensive evaluation report by using weighted scoring method; S54, the comprehensive evaluation report is pushed to the operation and maintenance end in real time through the visual dashboard, the translation accuracy threshold and the delay threshold are set, when the index is detected to be abnormal, the risk warning mechanism is automatically triggered, and the fault diagnosis report and targeted repair suggestion are generated.

9. The multi-lingual intelligent speech translation method for the call center system according to claim 8, wherein, The S51 comprises: According to the delay standard of each module in the efficiency requirement, the monitoring objects are ASR module, MT module, TTS module and data transmission delay between modules, the calculation method of each delay index is defined, and the whole link delay monitoring index list is generated; Real-time capture of each module processing delay and transmission time data, storage in monitoring database at second level frequency, real-time display of delay trend through Grafana dashboard, and generation of real-time delay data set; Compare the real-time delay data set, compare the actual delay of each module with the preset threshold, filter out the modules exceeding the threshold; at the same time, analyze the transmission time between modules, identify the redundant nodes with time consumption exceeding 50ms in the transmission path, and generate bottleneck node analysis report by summarizing the information of modules and redundant nodes exceeding the threshold. According to the high-load module / node information in the bottleneck node analysis report, an adaptive dynamic load balancing algorithm is used to set task allocation rules: the processing tasks of high-load modules are preferentially allocated to idle nodes with CPU occupancy rate < 30%, while the single-node task concurrency is limited, and a dynamic task allocation scheme is generated; Combined with the redundant transmission node information in the bottleneck node analysis report, the intermediate nodes without data processing function in the transmission path are deleted, the Websocket protocol in the open interface is used to replace the traditional HTTP protocol to transmit data, the data flow path between modules is re-planned, and an optimized transmission path diagram is generated; Collect the optimized full-link delay data, compare the delay changes before and after optimization, if there are still threshold delays, re-analyze the bottleneck and adjust the load balancing strategy or transmission path until the full-link delay meets the threshold requirements of each module, and generate an optimization effect verification report.

10. A call center system, characterized by The system comprises: One or more processors; a memory for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multi-language intelligent voice translation method of the call center system according to any one of claims 1 to 9.

Images