AI mobile phone communication man-machine coupling intelligent switching method based on personalized sound replication

Abstract

The application belongs to the technical field of communication, and discloses an AI mobile phone communication man-machine coupling intelligent switching method based on personalized sound replication, which comprises the following steps: S1, receiving a user's voice sample at an AI mobile phone terminal, extracting multi-dimensional voiceprint parameters, and constructing a personalized sound replication model; binding the personalized sound replication model with the user's communication identity identifier to form a communication sound configuration file; S2, when a communication task is triggered, the application scene of the communication task is discriminated based on the initiation mode and target object of the communication task, and a corresponding scene-based communication execution strategy is generated according to the discrimination result; S3, in the process of communication, the speech and semantics of the opposite end are identified in real time, the scene-based communication execution strategy is combined, and the intention of the opposite end and the service response state are jointly determined; the stability of the man-machine coupling communication process and the user experience are improved.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and more specifically, to an AI mobile communication human-machine coupling intelligent switching method based on personalized voice replication. Background Technology

[0002] AI voice interaction technology drives the application of AI mobile communication systems in scenarios such as intelligent outbound calls and customer service. To improve efficiency and meet complex interaction needs, existing technologies adopt an AI and human collaboration mode: AI automatically interacts during the call, and switches to human takeover when the complexity or risk increases, forming a human-machine coupled communication mode.

[0003] However, existing AI mobile communication human-machine coupling intelligent switching methods still have the following problems:

[0004] Existing handover methods have shortcomings: they rely on a single trigger condition (such as call timeout, keyword hit, etc.), only reflect instantaneous anomalies, and cannot assess the controllability and deterioration trend of the call from the overall time sequence, which can easily lead to erroneous handover, premature or delayed handover.

[0005] Secondly, communication status assessment relies on empirical rules or manual judgment, lacking unified and calculable quantitative indicators. This makes it difficult for the system to migrate parameters and maintain stable control across scenarios, hindering large-scale deployment and long-term operation. Thirdly, switching decisions are mostly triggered by discrete patterns, switching immediately when conditions are met. This fails to continuously characterize the degree of risk and differentiate risk levels, lacking quantitative basis and easily leading to frequent switching or sudden changes in user experience.

[0006] In view of this, the present invention proposes an AI mobile communication human-machine coupling intelligent switching method based on personalized voice replication to solve the above problems. Summary of the Invention

[0007] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: an AI mobile communication human-machine coupling intelligent switching method based on personalized voice replication, comprising: S1. Receive user voice samples on the AI ​​mobile terminal, extract multi-dimensional voiceprint parameters and build a personalized voice replication model; bind the personalized voice replication model with the user's communication identity to form a communication voice configuration profile; S2. When a communication task is triggered, the application scenario of the communication task is determined based on the initiation method and target object of the communication task, and a corresponding scenario-based communication execution strategy is generated according to the determination result. S3. During the call, the system identifies the other party's voice and semantics in real time, combines the scenario-based communication execution strategy, jointly determines the other party's intent and business response status, and determines the current dialogue node position and the next interaction goal, forming interaction status parameters. S4. Based on the interaction status parameters, call the corresponding personalized voice replication model in the communication voice configuration file to generate a lifelike voice output signal, and output it to the other end through the communication link; at the same time, continuously monitor the call interaction risk indicators and construct a switching judgment parameter set in combination with the interaction status parameters. S5. Evaluate the current communication status based on the switching judgment parameter set. When the communication status exceeds the controllable range of AI automatic interaction, generate a human takeover switching instruction. Under the premise of maintaining the original call connection without interruption, switch the communication control from AI to human.

[0008] Furthermore, the method for constructing a personalized voice replication model includes: The system receives voice samples uploaded by users through an AI mobile terminal, performs preprocessing on the acquired voice samples, including silence detection, endpoint segmentation, noise reduction filtering, and normalization, to form a voice sample sequence; based on the voice sample sequence, it extracts multi-dimensional voiceprint parameters, including spectral envelope parameters, fundamental frequency trajectory parameters, speech rate and rhythm parameters, and formant distribution parameters. Different types of multidimensional voiceprint parameters are mapped to a unified feature space, and a high-dimensional voiceprint feature vector for sound modeling is formed by feature concatenation. Through parameter fine-tuning training, the high-dimensional voiceprint feature vector is injected into the pre-trained speech generation basic model, thereby constructing a personalized voice replication model that highly matches the user's voice features.

[0009] Furthermore, the method for obtaining the communication voice configuration file includes: Obtain the communication identity identifier corresponding to the user and assign a unique identity index tag to each communication identity identifier; the communication identity identifier includes the user's main communication number, virtual communication number and in-application communication account; using the identity index tag as the association key, logically bind the personalized voice replication model with the corresponding user's communication identity identifier, establish the mapping relationship between communication identity and voice model, and form a communication voice configuration profile.

[0010] Furthermore, the method for obtaining the scenario-based communication execution strategy includes: When a communication task is triggered, the task attribute information of the communication task is obtained, including the initiation method of the communication task, the type of the target object, and the source of the communication trigger. Based on the task attribute information of the communication task, the application scenario of the communication task is determined, and the communication task is identified as one of the following scenarios: marketing notification scenario, transaction reminder scenario, service follow-up scenario, or emotional care scenario. For each application scenario, the corresponding script template, compliance control rules, and interaction strategy parameters are loaded from the preset scenario strategy library. The script template, compliance control rules, and interaction strategy parameters are then uniformly encapsulated to generate a scenario-based communication execution strategy corresponding to the current communication task.

[0011] Furthermore, the method for jointly determining the peer's intent and service response status includes: During the call, the system continuously collects the received voice signals from the other end and processes them into frames within a sliding time window to form continuous voice segments. The voice segments are transcribed in real time to obtain the corresponding text content. The text content is then semantically parsed to extract semantic feature information that represents the other end's intention. The semantic feature information is then jointly analyzed with the script flow template in the loaded scenario-based communication execution strategy. By judging the degree of conformity between the other end's voice response and the current script node, the system determines the other end's intention and business response status.

[0012] Furthermore, the method for obtaining the interaction state parameters includes: Based on the determination results of the peer's intent and business response status, the position of the current interaction node is determined, and the next interaction target connected to the current interaction node is inferred based on the interaction flow template. The obtained peer intent, business response status, position of the current interaction node, and next interaction target are uniformly encapsulated to form interaction status parameters used to characterize the progress of the current call interaction.

[0013] Furthermore, the method for generating near-human voice output includes: Based on the next interaction goal, the corresponding script content is determined from the preset script template library, and the script content is structured and parsed to divide the script text into different semantic units and label each semantic unit with the corresponding functional attributes. Based on the mapping relationship between preset functional attributes and speech generation parameters, speech generation control information including tone type, speech rate adjustment parameters, pause position markers and emphasis markers is generated; and the confidence score of the speech generation control information is read and used as the confidence level of the other end's semantic understanding. Threshold determination is made based on the confidence level of the peer's semantic understanding. When the confidence level of the peer's semantic understanding is greater than the preset confidence threshold of the peer's semantic understanding, it is determined that the peer's intention response is clear. The pause density is reduced and the default speech rate is adopted in the speech generation control information. When the confidence level of the peer's semantic understanding is less than or equal to the preset confidence threshold of the peer's semantic understanding, it is determined that there is uncertainty in the peer's intention response. A pause mark is added to the speech generation control information and a confirmation tone prompt is inserted. After adjusting the voice generation control information, the personalized voice replication model bound to the current communication identity is called from the communication voice configuration file. The speech content, the adjusted voice generation control information, and the user's personalized voiceprint parameters are used as joint inputs to generate a lifelike voice output signal that matches the current interaction state.

[0014] Furthermore, the method for constructing the switching determination parameter set includes: Monitor call interaction risk indicators that reflect abnormal call trends, including the number of times the other end interrupts abnormally, the cumulative number of semantic conflicts, and the number of times the interaction rhythm changes abnormally; based on the interaction status parameters and call interaction risk indicators, jointly evaluate the overall communication status of the current call, construct communication controllability evaluation parameters to characterize the current controllability of the communication, and map the communication controllability evaluation parameters to the communication status score. Within a continuous time window, the trend of communication status score changes is analyzed. When the communication status score is lower than the preset communication status score threshold and the communication status score shows a continuous downward trend in the time dimension, the current call is determined to have entered the risk out-of-control development range, and active intervention is triggered. After triggering the active intervention decision, the corresponding intervention intensity parameter is calculated based on the deviation difference between the communication status score and the target communication status score, as well as the magnitude of the rate of decrease in the communication status score. The communication controllability assessment parameters, the trend of communication status scores, and the intervention intensity parameters are structured and combined to form a set of handover decision parameters.

[0015] Furthermore, the method for evaluating the current communication state based on the handover determination parameter set includes: Based on the switching decision parameter set, the current communication status is continuously evaluated; when the communication status represented by the switching decision parameter set exceeds the preset controllable range of AI automatic interaction, it is determined that the current communication status is no longer suitable to be maintained independently by AI, and a human takeover switching instruction is generated simultaneously.

[0016] Furthermore, the method for generating the live human takeover switching instruction includes: While maintaining the original call connection, the communication control is switched from AI to human. Within the preset switching time window, the voice signal generated by AI and the voice signal of human are cross-faded and fused to gradually switch the communication control from AI to human.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention performs trend analysis on communication status scores within a continuous time window. The system only enters the active intervention zone when the communication status score is below a preset threshold and shows a continuous downward trend. This allows potential anomalies to be identified in advance before the risk is completely out of control, avoiding switching based on instantaneous or single anomalies, and significantly improving the foresight and stability of communication control.

[0018] By structurally combining communication controllability assessment parameters, communication status score change trend parameters, and intervention intensity parameters, a handover decision parameter set is constructed. This enables the generation of human takeover handover commands based on multi-dimensional and time-series comprehensive judgment, reducing the probability of erroneous and excessive handovers. The system behavior exhibits good interpretability, facilitating flexible configuration according to different application scenarios, user preferences, or risk levels, thereby enhancing engineering implementation and deployment capabilities.

[0019] During the handover of control, the system maintains parallel AI and human voice links within the same communication session channel and achieves continuous fusion of voice signals through a smoothing weight function, ensuring the call connection remains valid and avoiding interruptions caused by reconnection or transfer. Simultaneously, it guarantees the continuity of voice energy, rhythm, and auditory quality during the handover process, reducing the other end's perception of the handover behavior. By preset the handover start time and fade-out transition duration, the handover rhythm can be flexibly adjusted according to the scenario, making the transfer of control smooth and controllable, with good reproducibility and cross-platform adaptability. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of the AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication of the present invention. Figure 2 This is a schematic diagram of the AI ​​mobile communication human-machine coupling intelligent switching system based on personalized voice replication of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1 Please see Figure 1 As shown, this embodiment provides an AI mobile communication human-machine coupling intelligent switching method based on personalized voice replication, specifically including the following steps: S1. Receive user voice samples on the AI ​​mobile terminal, extract multi-dimensional voiceprint parameters and build a personalized voice replication model; bind the personalized voice replication model with the user's communication identity to form a communication voice configuration profile; S2. When a communication task is triggered, the application scenario of the communication task is determined based on the initiation method and target object of the communication task, and a corresponding scenario-based communication execution strategy is generated according to the determination result. S3. During the call, the system identifies the other party's voice and semantics in real time, combines the scenario-based communication execution strategy, jointly determines the other party's intent and business response status, and determines the current dialogue node position and the next interaction goal, forming interaction status parameters. S4. Based on the interaction status parameters, call the corresponding personalized voice replication model in the communication voice configuration file to generate a lifelike voice output signal, and output it to the other end through the communication link; at the same time, continuously monitor the call interaction risk indicators and construct a switching judgment parameter set in combination with the interaction status parameters. S5. Evaluate the current communication status based on the switching judgment parameter set. When the communication status exceeds the controllable range of AI automatic interaction, generate a human takeover switching instruction. Under the premise of maintaining the original call connection without interruption, switch the communication control from AI to human.

[0023] Methods for building personalized voice replication models include: The AI ​​mobile terminal receives voice samples uploaded by users. The voice samples are pre-recorded voice files, including natural dialogue sentences, sentences with emotional changes, and standard phoneme combinations covering multiple pronunciation positions, in order to ensure the diversity of voice samples in dimensions such as pitch, speech rate, emotional state and speech coherence. For example: capturing natural conversational phrases from users' daily lives, such as "Hello, how can I help you?", "Please wait a moment, let me look into it," "Please repeat that, thank you," etc. Acquire emotional changes in user statements with different sentiments during natural conversations, such as "The weather is nice today" (neutral); "Thank you for your help" (gentle); "Please reply to me as soon as possible" (urgent). The acquired speech samples are preprocessed, including silence detection, endpoint segmentation, noise reduction filtering, and normalization, to form a speech sample sequence. Based on the speech sample sequence, multi-dimensional voiceprint parameters for characterizing the user's voice personality are extracted, including spectral envelope parameters, fundamental frequency trajectory parameters, speech rate and rhythm parameters, and formant distribution parameters. Among them, short-time Fourier transform or cepstral analysis is performed on speech sample sequences to extract the energy distribution contours of speech samples in different frequency ranges, forming spectral envelope parameters that describe the overall shape of the spectrum. These parameters are used to characterize the user's vocal tract structure and timbre characteristics, and are the basic parameters for distinguishing the voice personalities of different speakers, providing stable timbre constraints for the voice replication model. By performing periodic analysis or autocorrelation analysis on the speech sample sequence, the fundamental frequency value of each time frame is extracted, and the change of the fundamental frequency over time is continuously modeled to form the fundamental frequency trajectory parameter, which is used to reflect the pitch characteristics and intonation habits of the user when speaking, so that the generated speech is closer to the user's real voice in terms of intonation. By statistically analyzing the number of syllables per unit time, syllable duration, and pause intervals between adjacent syllables in a speech sample sequence, speech rate and rhythm parameters are obtained to characterize the user's speaking rhythm and expression habits, so that the speech output has personalized features in terms of speed, rhythm, and pause arrangement, avoiding the mechanical generation of speech. By using linear predictive analysis, the formant distribution positions of speech sample sequences are identified, and formant distribution parameters are obtained to characterize users' pronunciation habits, playing a key supplementary role in fine-grained timbre differentiation and high-similarity sound replication. It should be noted that silence segment detection is used to identify time segments in speech samples that do not contain effective vocal information, such as pauses before and after speaking, breathing intervals, or ambient noise segments. It determines whether a time segment belongs to a silence segment by analyzing the short-time energy and zero-crossing rate of the speech sample. When the short-time energy is lower than the preset energy threshold and the duration exceeds the minimum silence duration, it is determined to be a silence segment. Endpoint segmentation, based on the results of silence segment detection, determines the start and end points of each valid speech segment, thereby splitting continuously recorded long speech into several speech segments. Through endpoint segmentation, invalid interference between different sentences can be avoided, and each speech segment corresponds to a complete vocal behavior. For example, if a speech sample contains three independent sentences, endpoint segmentation can be used to separate the speech sample into three speech segments, which can be used for subsequent feature statistics, preventing pauses at sentence boundaries from affecting rhythm analysis. Denoising filtering is used to reduce the impact of environmental noise, background current noise, or equipment noise on speech signals. Specifically, frequency domain filtering, spectral subtraction, or adaptive noise estimation methods can be used to suppress non-human voice frequency bands, so that the retained speech signal is concentrated in the effective frequency range of human voice. Normalization processing is used to eliminate the influence of differences in volume and energy distribution under different recording conditions. By uniformly scaling the amplitude of the speech signal, features can be extracted from different speech samples within the same energy range. Different types of multidimensional voiceprint parameters are mapped to a unified feature space, and a high-dimensional voiceprint feature vector for sound modeling is formed by feature concatenation. Through parameter fine-tuning training, the high-dimensional voiceprint feature vector is injected into the pre-trained speech generation basic model, thereby constructing a personalized voice replication model that highly matches the user's voice features.

[0024] It should be noted that the pre-trained speech generation basic model consists of three parts: encoder, decoder, and Mel spectrum generator. First, the speech generation basic model is pre-trained based on a publicly available large-scale speech dataset (containing 10,000 user speech samples of different ages, genders, and dialects, with a total duration of 1,000 hours). During pre-training, the Adam optimizer was used with an initial learning rate of 1e-4, a batch size of 32, and 100 training epochs. Mel spectral distortion and speech naturalness scores were used as loss functions. Pre-training was stopped when the loss function value stabilized (the loss value fluctuated ≤0.001 for 10 consecutive training epochs), resulting in a converged basic speech generation model that can initially replicate general speech, but with low personalization. Parameter fine-tuning training: The high-dimensional voiceprint feature vector is associated with the corresponding speech sample sequence to form a fine-tuning dataset; at the same time, in order to improve the generalization ability of the model, data augmentation technology is used to expand the fine-tuning dataset: by changing the pitch (±5%), speech rate (±10%) and volume (±15%) of the speech samples, three times the original data augmented samples are generated, and finally a personalized fine-tuning dataset is obtained. The personalized fine-tuning dataset is input into the pre-trained speech generation base model for parameter fine-tuning training. During the fine-tuning process, the parameters of the first 3 layers of the encoder are frozen (to retain the ability to extract general speech features), and only the parameters of the last 2 layers of the encoder, the decoder, and the Mel spectrum generator are fine-tuned. The optimizer is still Adam, and the learning rate is adjusted to 1e-5 (lower than the pre-training learning rate to avoid damaging the performance of the speech generation base model), the batch size is 8, and the training epochs are 30. The loss function used in the fine-tuning process is a combination of "Mel spectrum distortion + voiceprint similarity loss". The voiceprint similarity loss is obtained by calculating the cosine similarity between the multidimensional voiceprint parameters of the model output speech and the user's real multidimensional voiceprint parameters. When the value of the combined loss function fluctuates ≤0.0005 for 5 consecutive training rounds and the similarity between the model output speech and the user's original speech is ≥92%, the fine-tuning is stopped, and the trained personalized voice replication model is obtained.

[0025] Methods for obtaining communication voice configuration files include: Obtain the communication identity identifier corresponding to the user and assign a unique identity index tag to each communication identity identifier; the communication identity identifier includes the user's main communication number, virtual communication number, and in-application communication account; using the identity index tag as the association key, logically bind the personalized voice replication model to the corresponding user's communication identity identifier, establish a mapping relationship between communication identity and voice model, and form a communication voice configuration file; the communication voice configuration file is used to specify the personalized voice replication model corresponding to different communication identity identifiers when they are invoked, so as to avoid the mixing of voices in multiple identity scenarios within the same terminal.

[0026] Methods for obtaining scenario-based communication execution strategies include: When a communication task is triggered, the task attribute information of the communication task is obtained, including the initiation method of the communication task, the target object type, and the communication trigger source; among which, the initiation method of the communication task includes manual triggering, automatic triggering by the business system, or triggering by timed rules; the target object type includes customers, internal personnel, or preset contact groups; Based on the task attribute information of the communication task, the application scenario of the communication task is identified, and the communication task is determined to be one of the following scenarios: marketing notification scenario, task reminder scenario, service follow-up scenario, or emotional care scenario. For example, when a communication task is automatically triggered and its attribute information contains keywords such as promotion, discount, or event notification, the system determines that the communication task belongs to a marketing notification scenario; when a communication task contains deterministic transaction content such as bill reminders, appointment confirmations, or status change notifications, the system determines that it belongs to a transaction reminder scenario; when a communication task is used to obtain feedback or confirm satisfaction after existing services are completed, it is determined to be a service follow-up scenario; and when the task attribute information of a communication task indicates that its main purpose is to greet, care for, or soothe emotions, it is determined to be an emotional care scenario.

[0027] For each application scenario, the corresponding script template, compliance control rules, and interaction strategy parameters are loaded from the preset scenario strategy library. The preset scenario strategy library can be pre-built by staff based on expert experience. The dialogue flow template includes multiple dialogue nodes arranged in the order of interaction. Each dialogue node contains at least standard dialogue content, variable parameter placeholders, and conditions for the next dialogue branch, which are used to guide the dialogue progress during the communication process. Compliance control rules are used to limit the scope of language used in communication, outbound call time windows, conditions for reaching sensitive information, and methods for handling user rejection responses, so as to ensure that the communication process complies with business specifications and compliance requirements; interaction strategy parameters include voice emotion style parameters, speech rate adjustment parameters, and interaction initiative level parameters, which are used to constrain the AI's interaction methods and response intensity during communication. The script flow template, compliance control rules, and interaction strategy parameters are uniformly encapsulated to generate a scenario-based communication execution strategy corresponding to the current communication task.

[0028] Methods for jointly determining the intent and business response status of the other end include: During the call, the system continuously collects the received voice signal from the other end and processes it into frames within a sliding time window to form continuous voice segments; the voice segments are then transcribed in real time to obtain the corresponding text content. Semantic analysis of text content is performed, including keyword extraction, semantic vector encoding, and sentiment analysis, to extract semantic features that represent the other party's intention. These semantic features include intention category and sentiment label. Keyword extraction is used to identify trigger words or negation words in the text content. Semantic vector encoding is used to map the text content into comparable semantic feature representations. Sentiment analysis is used to determine the emotional state expressed by the text content. The semantic feature information is jointly analyzed with the speech flow template in the loaded scenario-based communication execution strategy. By judging the degree of conformity between the peer's voice response and the current speech node, the peer's intent and business response status are determined. The business response status is one of valid response, rejection response, or abnormal response.

[0029] For example: when the compliance level reaches a preset compliance threshold, the business response status is determined to be a valid response; when the compliance level is lower than the preset compliance threshold or negative semantic features appear, the business response status is determined to be a rejected response; when abnormal or unexpected semantics are detected, the business response status is determined to be an abnormal response. Methods for obtaining interaction status parameters include: Based on the determination results of the peer's intent and business response status, the position of the current interaction node is determined, and the next interaction target connected to the current interaction node is inferred based on the interaction flow template to guide the subsequent communication interaction process. The obtained peer intent, business response status, position of the current interaction node, and next interaction target are uniformly encapsulated to form interaction status parameters used to characterize the progress of the current call interaction.

[0030] Methods for generating lifelike speech output include: Based on the next interaction goal, the corresponding dialogue content is determined from the preset dialogue template library, and the dialogue content is structured and parsed, dividing the dialogue text into different semantic units and labeling each semantic unit with corresponding functional attributes; a set of structured dialogue resources can be pre-built based on expert experience, communication task scenarios and historical communication data to form a preset dialogue template library. Based on the mapping relationship between preset functional attributes and speech generation parameters, speech generation control information including tone type, speech rate adjustment parameters, pause position markers and emphasis markers is generated; and the confidence score of the speech generation control information is read and used as the confidence level of the other end's semantic understanding. Threshold determination is made based on the confidence level of the peer's semantic understanding. When the confidence level of the peer's semantic understanding is greater than the preset confidence threshold of the peer's semantic understanding, it is determined that the peer's intention response is clear. The pause density is reduced and the default speech rate is adopted in the speech generation control information. When the confidence level of the peer's semantic understanding is less than or equal to the preset confidence threshold of the peer's semantic understanding, it is determined that there is uncertainty in the peer's intention response. A pause mark is added to the speech generation control information and a confirmation tone prompt is inserted. After adjusting the voice generation control information, the personalized voice replication model bound to the current communication identity is called from the communication voice configuration file. The speech content, the adjusted voice generation control information, and the user's personalized voiceprint parameters are used as joint inputs to generate a lifelike voice output signal that matches the current interaction state. Before outputting the lifelike voice output signal, the lifelike voice output signal is aligned with the real-time voice interaction timing of the current call, so that the lifelike voice output signal is sent to the other end through the communication link when the other end finishes the voice call.

[0031] Methods for constructing the switching decision parameter set include: Monitor call interaction risk indicators that reflect abnormal call trends, including the number of times abnormal interruptions occur from the other end, the cumulative number of semantic conflicts, and the number of times abnormal changes occur in the interaction rhythm; based on the interaction status parameters and call interaction risk indicators, jointly evaluate the overall communication status of the current call, construct communication controllability assessment parameters to characterize the controllability of the current communication, and map the communication controllability assessment parameters to communication status scores to reflect whether the current call is still within the predictable and convergent control range of AI automatic interaction; The communication status score is: ;in, Indicates time The communication status score is used to quantify the controllability of the current communication and comprehensively reflect whether the AI ​​can still perform stable, predictable, and low-risk automatic interactive control of the call. Indicates time Interaction state parameters; Indicates time Call interaction risk indicators; An index representing a time point; Within a continuous time window, the trend of communication status score changes is analyzed. When the communication status score is lower than the preset communication status score threshold and the communication status score shows a continuous downward trend in the time dimension, the current call is determined to have entered the risk out-of-control development range, and active intervention is triggered. Active intervention is determined as follows: ;in, This represents the active intervention trigger flag, a logical judgment variable used to indicate whether the system has entered a state requiring active intervention and evaluation. A value of 1 indicates that the intervention trigger condition is met and the system enters the active intervention range; a value of 0 indicates that the condition is not met and normal AI interaction continues. This indicates the preset communication status scoring threshold; This represents the rate of change of communication status, used to characterize the trend and direction of the communication status score over time. This represents a trend indicator used to indicate whether the current call communication status is in a continuously deteriorating trend. It takes a value of 1 when the communication status score decreases over time, and a value of 0 otherwise. After triggering the active intervention decision, the corresponding intervention intensity parameter is calculated based on the deviation difference between the communication status score and the target communication status score, as well as the magnitude of the rate of decrease of the communication status score. This quantifies the degree of intervention required for the current call, transforming the intervention process from a discrete decision on whether to switch to a continuously adjustable intensity assessment process. The intervention intensity parameters are: ;in, This parameter represents the intervention intensity, quantifying the level of intervention required for the current call. This represents the target communication state score, which is the ideal communication state level that the preset system expects to maintain. This represents the intervention sensitivity coefficient, used to adjust the system's sensitivity to changes in communication status; The magnitude of the rate of change of communication status is used to characterize how fast the communication status score changes over time, that is, the degree of drastic change in communication status. The communication controllability assessment parameters, the trend of communication status scores, and the intervention intensity parameters are structured and combined to form a handover decision parameter set, which is then used as the quantitative decision-making basis for generating human takeover handover instructions.

[0032] Methods for evaluating the current communication state based on a handover decision parameter set include: Based on the switching decision parameter set, the current communication status is continuously evaluated; when the communication status represented by the switching decision parameter set exceeds the preset controllable range of AI automatic interaction, it is determined that the current communication status is no longer suitable to be maintained independently by AI, and a human takeover switching instruction is generated simultaneously.

[0033] It should be noted that the process of continuously evaluating the current communication status based on the handover decision parameter set includes the following: During the call interaction, the system continuously updates the handover decision parameter set according to a preset time granularity. The handover decision parameter set consists of communication controllability assessment parameters, communication status score, the trend of the communication status score, and intervention intensity parameters, which are used to characterize the controllability of the current communication status from multiple dimensions. Within each evaluation cycle, the system performs a joint analysis of the above parameters to determine whether the communication status is stable, adjustable, or trending out of control. This analysis not only examines the instantaneous values ​​of each parameter but also comprehensively considers the temporal direction and magnitude of the communication status score, thereby achieving continuous and dynamic evaluation of the communication status.

[0034] The preset controllable range of AI automatic interaction is determined in advance through statistical analysis of historical call data, verified scenarios where AI can independently maintain a call, and interaction state parameters under different risk levels. This range is used to limit the parameter interval within which AI can effectively control the communication state in automatic interaction mode. This controllable range corresponds to the allowed value combination of communication controllability assessment parameters, communication state scores and their changing trends, and intervention intensity parameters. When the set of switching judgment parameters obtained from continuous evaluation exceeds the controllable range in whole or in part, indicating that AI is unable to restore the communication state to the target controllable range within a reasonable time, the system determines that the current communication state has exceeded the controllable range of AI automatic interaction, and uses this as the basis for triggering a human takeover switching command.

[0035] Methods for generating live human takeover switching instructions include: While maintaining the original call connection without interruption, the communication control is switched from AI to human. During the switching of communication control, the effective connection between the AI ​​voice output link and the human voice input link is maintained simultaneously within the same communication session channel. Within a preset switching time window, the AI-generated voice signal and the real human voice signal are cross-faded and fused to obtain the final voice signal. This causes the output weight of the AI-generated voice signal to gradually decrease over time, while the output weight of the real human voice signal increases synchronously, thus gradually switching the communication control from AI to the real human.

[0036] Cross-fading and fusion processing: ;in, Indicates at time The final voice signal output to the communication link; The smoothing weight function represents the fusion weight that changes over time. It is used to control the ratio of AI voice to human voice during the switching process. Its value changes continuously between 0 and 1 to achieve a smooth transition of voice. Indicates at time AI-generated speech signals output by the AI ​​speech generation module; Indicates at time Real-time voice signals collected and output by a real person after being taken over; Smoothing weight function ;in, This indicates the starting point at which the cross-fading and fusion processing of AI-generated speech signals and human speech signals begins. This indicates the time required for a smooth transition from AI voice to human voice. The preset energy threshold is set by staff by collecting different short-term energy values ​​and taking the average of multiple short-term energy values ​​as the preset energy threshold; similarly, preset compliance threshold, preset peer semantic understanding confidence threshold, and preset communication status scoring threshold are set.

[0037] In this embodiment, by performing trend analysis on the communication status score within a continuous time window, the system only enters the active intervention zone when the communication status score is lower than a preset threshold and shows a continuous downward trend. This allows for the early identification of potential anomalies before the risk is completely out of control, avoiding switching based on instantaneous or single anomalies, and significantly improving the foresight and stability of communication control.

[0038] By structurally combining communication controllability assessment parameters, communication status score change trend parameters, and intervention intensity parameters, a handover decision parameter set is constructed. This enables the generation of human takeover handover commands based on multi-dimensional and time-series comprehensive judgment, reducing the probability of erroneous and excessive handovers. The system behavior exhibits good interpretability, facilitating flexible configuration according to different application scenarios, user preferences, or risk levels, thereby enhancing engineering implementation and deployment capabilities.

[0039] During the handover of control, the system maintains parallel AI and human voice links within the same communication session channel and achieves continuous fusion of voice signals through a smoothing weight function, ensuring the call connection remains valid and avoiding interruptions caused by reconnection or transfer. Simultaneously, it guarantees the continuity of voice energy, rhythm, and auditory quality during the handover process, reducing the other end's perception of the handover behavior. By preset the handover start time and fade-out transition duration, the handover rhythm can be flexibly adjusted according to the scenario, making the transfer of control smooth and controllable, with good reproducibility and cross-platform adaptability.

[0040] Example 2 Please see Figure 2 As shown, for parts not described in detail in this embodiment, please refer to the description in Embodiment 1. An AI-based mobile communication human-machine coupling intelligent switching system based on personalized voice replication is provided, including: The voice configuration module receives user voice samples on the AI ​​mobile terminal, extracts multi-dimensional voiceprint parameters and constructs a personalized voice replication model; it then binds the personalized voice replication model with the user's communication identity to form a communication voice configuration profile. The scenario discrimination module, when a communication task is triggered, performs application scenario discrimination based on the initiation method and target object of the communication task, and generates a corresponding scenario-based communication execution strategy based on the discrimination result; The interaction recognition module identifies the other party's voice and semantics in real time during the call. Combined with the contextualized communication execution strategy, it jointly determines the other party's intent and business response status, and determines the current dialogue node position and the next interaction target, forming interaction status parameters. The voice generation module, based on the interaction state parameters, calls the corresponding personalized voice replication model in the communication voice configuration file to generate a lifelike voice output signal, which is then output to the other end through the communication link; at the same time, it continuously monitors call interaction risk indicators and constructs a switching judgment parameter set in combination with the interaction state parameters. The human-machine coupling module evaluates the current communication status based on the switching judgment parameter set. When the communication status exceeds the controllable range of AI automatic interaction, it generates a human takeover switching command, which switches the communication control from AI to human while maintaining the original call connection.

[0041] Since the electronic device described in this embodiment is the one used to implement the AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication in the embodiments of this application, those skilled in the art can understand the specific implementation methods and various variations of the electronic device in this embodiment based on the AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any electronic device used by those skilled in the art to implement the AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication in the embodiments of this application falls within the scope of protection of this application.

[0042] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0043] The above description is merely a preferred embodiment of the present invention, and the scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for users of ordinary technical skills, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An AI-powered mobile communication human-machine coupling intelligent switching method based on personalized voice replication, characterized in that, include: S1. Receive user voice samples on the AI ​​mobile terminal, extract multi-dimensional voiceprint parameters and build a personalized voice replication model; bind the personalized voice replication model with the user's communication identity to form a communication voice configuration profile; S2. When a communication task is triggered, the application scenario of the communication task is determined based on the initiation method and target object of the communication task, and a corresponding scenario-based communication execution strategy is generated according to the determination result. S3. During the call, the system identifies the other party's voice and semantics in real time, combines the scenario-based communication execution strategy, jointly determines the other party's intent and business response status, and determines the current dialogue node position and the next interaction goal, forming interaction status parameters. S4. Based on the interaction status parameters, call the corresponding personalized voice replication model in the communication voice configuration file to generate a lifelike voice output signal, and output it to the other end through the communication link; at the same time, continuously monitor the call interaction risk indicators and construct a switching judgment parameter set in combination with the interaction status parameters. S5. Evaluate the current communication status based on the switching judgment parameter set. When the communication status exceeds the controllable range of AI automatic interaction, generate a human takeover switching instruction. Under the premise of maintaining the original call connection without interruption, switch the communication control from AI to human.

2. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 1, characterized in that, The method for constructing a personalized voice replication model includes: The system receives voice samples uploaded by users through an AI mobile terminal, performs preprocessing on the acquired voice samples, including silence detection, endpoint segmentation, noise reduction filtering, and normalization, to form a voice sample sequence; based on the voice sample sequence, it extracts multi-dimensional voiceprint parameters, including spectral envelope parameters, fundamental frequency trajectory parameters, speech rate and rhythm parameters, and formant distribution parameters. Different types of multidimensional voiceprint parameters are mapped to a unified feature space, and a high-dimensional voiceprint feature vector for sound modeling is formed by feature concatenation. Through parameter fine-tuning training, the high-dimensional voiceprint feature vector is injected into the pre-trained speech generation basic model, thereby constructing a personalized voice replication model that highly matches the user's voice features.

3. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 2, characterized in that, The method for obtaining the communication voice configuration file includes: Obtain the communication identity identifier corresponding to the user and assign a unique identity index tag to each communication identity identifier; the communication identity identifier includes the user's main communication number, virtual communication number and in-application communication account; using the identity index tag as the association key, logically bind the personalized voice replication model with the corresponding user's communication identity identifier, establish the mapping relationship between communication identity and voice model, and form a communication voice configuration profile.

4. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 3, characterized in that, The method for obtaining the scenario-based communication execution strategy includes: When a communication task is triggered, the task attribute information of the communication task is obtained, including the initiation method of the communication task, the type of the target object, and the source of the communication trigger. Based on the task attribute information of the communication task, the application scenario of the communication task is determined, and the communication task is identified as one of the following scenarios: marketing notification scenario, transaction reminder scenario, service follow-up scenario, or emotional care scenario. For each application scenario, the corresponding script template, compliance control rules, and interaction strategy parameters are loaded from the preset scenario strategy library. The script template, compliance control rules, and interaction strategy parameters are then uniformly encapsulated to generate a scenario-based communication execution strategy corresponding to the current communication task.

5. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 4, characterized in that, The method for jointly determining the intent of the peer and the service response status includes: During the call, the system continuously collects the received voice signals from the other end and processes them into frames within a sliding time window to form continuous voice segments. The voice segments are transcribed in real time to obtain the corresponding text content. The text content is then semantically parsed to extract semantic feature information that represents the other end's intention. The semantic feature information is then jointly analyzed with the script flow template in the loaded scenario-based communication execution strategy. By judging the degree of conformity between the other end's voice response and the current script node, the system determines the other end's intention and business response status.

6. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 5, characterized in that, The methods for obtaining the interaction state parameters include: Based on the determination results of the peer's intent and business response status, the position of the current interaction node is determined, and the next interaction target connected to the current interaction node is inferred based on the interaction flow template. The obtained peer intent, business response status, position of the current interaction node, and next interaction target are uniformly encapsulated to form interaction status parameters used to characterize the progress of the current call interaction.

7. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 6, characterized in that, The method for generating near-human voice output includes: Based on the next interaction goal, the corresponding script content is determined from the preset script template library, and the script content is structured and parsed to divide the script text into different semantic units and label each semantic unit with the corresponding functional attributes. Based on the mapping relationship between preset functional attributes and speech generation parameters, speech generation control information including tone type, speech rate adjustment parameters, pause position markers and emphasis markers is generated; and the confidence score of the speech generation control information is read and used as the confidence level of the other end's semantic understanding. Threshold determination is made based on the confidence level of the peer's semantic understanding. When the confidence level of the peer's semantic understanding is greater than the preset confidence threshold of the peer's semantic understanding, it is determined that the peer's intention response is clear. The pause density is reduced and the default speech rate is adopted in the speech generation control information. When the confidence level of the peer's semantic understanding is less than or equal to the preset confidence threshold of the peer's semantic understanding, it is determined that there is uncertainty in the peer's intention response. A pause mark is added to the speech generation control information and a confirmation tone prompt is inserted. After adjusting the voice generation control information, the personalized voice replication model bound to the current communication identity is called from the communication voice configuration file. The speech content, the adjusted voice generation control information, and the user's personalized voiceprint parameters are used as joint inputs to generate a lifelike voice output signal that matches the current interaction state.

8. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 7, characterized in that, The method for constructing the switching determination parameter set includes: Monitor call interaction risk indicators that reflect abnormal call trends, including the number of times the other end interrupts abnormally, the cumulative number of semantic conflicts, and the number of times the interaction rhythm changes abnormally; based on the interaction status parameters and call interaction risk indicators, jointly evaluate the overall communication status of the current call, construct communication controllability evaluation parameters to characterize the current controllability of the communication, and map the communication controllability evaluation parameters to the communication status score. Within a continuous time window, the trend of communication status score changes is analyzed. When the communication status score is lower than the preset communication status score threshold and the communication status score shows a continuous downward trend in the time dimension, the current call is determined to have entered the risk out-of-control development range, and active intervention is triggered. After triggering an active intervention decision, the corresponding intervention intensity parameter is calculated based on the deviation between the communication status score and the target communication status score, as well as the rate of decrease in the communication status score. The communication controllability assessment parameters, the trend of communication status scores, and the intervention intensity parameters are structured and combined to form a set of handover decision parameters.

9. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 8, characterized in that, The method for evaluating the current communication state based on the handover decision parameter set includes: Based on the switching decision parameter set, the current communication status is continuously evaluated; when the communication status represented by the switching decision parameter set exceeds the preset controllable range of AI automatic interaction, it is determined that the current communication status is no longer suitable to be maintained independently by AI, and a human takeover switching instruction is generated simultaneously.

10. The AI ​​mobile communication human-machine coupling intelligent switching method based on personalized voice replication according to claim 9, characterized in that, The method for generating the live human takeover switching command includes: While maintaining the original call connection, the communication control is switched from AI to human. Within the preset switching time window, the voice signal generated by AI and the voice signal of human are cross-faded and fused to gradually switch the communication control from AI to human.

Images