Multi-model hierarchical screening and knowledge-enhanced ai mobile phone intelligent communication interaction method

Abstract

The application provides a multi-model hierarchical screening and knowledge-enhanced AI mobile phone intelligent communication interaction method, relates to the technical field of artificial intelligence, and through the construction of a multi-model hierarchical screening mechanism, performs intent recognition and confidence score on the multi-modal initial data of a call, dynamically routes the call to a B-end processing channel, a C-end processing channel or a mixed processing channel according to the score result, realizes real-time shunting and accurate judgment of business attributes and emotional attributes, and constructs a multi-target loss function containing a business target loss item, an emotional target loss item and a language fluency loss item, iteratively generates response content with the minimum loss function as the target, so that the generated content meets the compliance and conversion rate requirements of enterprise business, and meets the emotional expectations and personalized expression needs of users, collects interaction result data as a reward signal for online optimization, and realizes the cooperative output of business targets and emotional values in one call.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to an AI-powered mobile phone intelligent communication and interaction method that employs multi-model hierarchical screening and knowledge enhancement. Background Technology

[0002] With the deep integration of artificial intelligence (AI) technology and mobile communications, smartphones are gradually evolving into AI phones with on-device intelligent processing capabilities. In communication interaction scenarios, AI phones not only need to handle traditional voice call functions but also provide business services such as automated marketing and notification follow-ups for enterprise clients, while offering personalized services such as schedule reminders and emotional support to individual users. However, existing AI communication solutions generally suffer from a lack of focus. Intelligent outbound calling systems for enterprises focus on improving marketing efficiency and conversion rates, achieving business outreach through pre-set script templates and batch outbound calls, but lack attention to users' personalized emotional needs. Voice assistants for individuals focus on understanding user commands, providing information queries and daily reminders, but cannot fulfill the business objectives of enterprise clients. When enterprises need to deliver business information to individual users through AI phones, existing technologies cannot simultaneously address the enterprise's compliance and conversion rate requirements with users' personalized and emotional expectations in the same call. This often leads to commercial marketing calls being perceived as harassment by users, while emotional support services fail to achieve commercial value conversion. Therefore, how to achieve a synergistic balance between business efficiency and emotional experience on AI phones as an on-device device has become a pressing technical problem to be solved in this field.

[0003] Existing AI mobile phone communication interaction methods suffer from the technical deficiency of having a single objective, failing to simultaneously meet the business objectives of enterprises and the emotional experience needs of users during the same call. This results in a disconnect between commercial marketing and emotional companionship, making it difficult to achieve a win-win situation for both parties. Therefore, this invention proposes an AI mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement to solve the above problems. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: an AI-powered mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement, the method comprising: Acquire multimodal initial data of the call, extract and fuse features from the multimodal initial data, and output the intent label of the call and the corresponding confidence score; Based on the intent label and confidence score, a routing decision is made to route the call to the corresponding processing channel, which includes a B-end processing channel, a C-end processing channel and a hybrid processing channel. By leveraging the enterprise knowledge graph and star IP knowledge vector library within the post-routing processing channel, combined with intent tags, retrieval and matching are performed to generate enhanced recall information. The intent tags and recall enhancement information are input into a multi-objective loss function for solution to generate response content; The response content is output, and the final result data of this interaction is collected as a reward signal. Based on the reward signal, the intent label generation process and multi-objective loss function are optimized to complete the intelligent communication interaction.

[0005] Furthermore, the acquisition of multimodal initial data for the call includes: Obtain the calling number, the called number, and the number authentication label; The system acquires the audio segment at the start of the call and performs real-time transcription of the audio segment to obtain the real-time transcribed text. Get the timestamp information of the call.

[0006] Furthermore, the step of extracting and fusing features from the initial multimodal data to output the intent label of the call and the corresponding confidence score includes: Feature extraction is performed on the multimodal initial data to obtain number feature vector, voice feature vector and text feature vector; The number feature vector, voice feature vector, and text feature vector are fused using multimodal feature fusion to obtain a fused feature vector; Based on the fused feature vector, intent classification calculation is performed to obtain the initial score corresponding to each preset intent category; The initial scores are normalized to obtain confidence scores for each preset intent category. The preset intent categories include a business intent sub-label set, an emotional intent sub-label set, and a hybrid intent label set. The confidence scores include business intent confidence scores, emotional intent confidence scores, and hybrid intent confidence scores.

[0007] Furthermore, the step of making routing decisions based on the intent tag and confidence score, and routing the call to the corresponding processing channel, includes: When the confidence score of business intent is higher than the first preset threshold and the confidence score of sentiment intent is lower than the second preset threshold, the call will be routed to the B-end processing channel. When the confidence score of emotional intent is higher than the second preset threshold and the confidence score of business intent is lower than the first preset threshold, the call will be routed to the C-end processing channel. When both the confidence scores for business intent and sentiment intent are higher than their respective lower thresholds, or both are lower than the first and second preset thresholds, the call will be routed to the hybrid processing channel.

[0008] Furthermore, the process utilizes the enterprise knowledge graph and celebrity IP knowledge vector library within the post-routing processing channel, combined with intent tags, for retrieval and matching to generate enhanced recall information, including: When routing to the B-end processing channel, obtain the enterprise identifier associated with the call; Based on the enterprise identifier, the corresponding enterprise knowledge graph is read. The enterprise knowledge graph is constructed with the enterprise entity as the central node, and includes a graph structure including standard language nodes, product knowledge nodes, user profile nodes, compliance rule nodes and related relationships. Using the enterprise identifier and intent tag as query conditions, retrieve the nodes and paths associated with the current intent tag in the enterprise knowledge graph to obtain candidate knowledge fragments; The candidate knowledge fragments are sorted and filtered, and one or more fragments with the highest matching degree with the intent tag are selected and combined to generate recall enhancement information.

[0009] Furthermore, the process utilizes the enterprise knowledge graph and celebrity IP knowledge vector library within the post-routing processing channel, combined with intent tags, for retrieval and matching to generate enhanced recall information, including: When routing to the C-end processing channel, obtain the user identifier associated with the call; Based on the user identifier, the corresponding celebrity IP knowledge vector library is read. The celebrity IP knowledge vector library is a multimodal vector library constructed based on user-authorized celebrity voice samples, image pictures, catchphrases, and historical interaction records between users and celebrities. Using the user identifier and intent tag as query conditions, calculate the similarity between the query and each vector in the celebrity IP knowledge vector library, and select the knowledge entries corresponding to one or more vectors whose similarity exceeds a preset vector threshold. The content from the selected knowledge items is combined to generate recall enhancement information.

[0010] Furthermore, the process utilizes the enterprise knowledge graph and celebrity IP knowledge vector library within the post-routing processing channel, combined with intent tags, for retrieval and matching to generate enhanced recall information, including: When routed to the hybrid processing channel, both enterprise knowledge graph-based retrieval and celebrity IP knowledge vector database-based retrieval are performed simultaneously to obtain enterprise knowledge retrieval results and celebrity IP knowledge retrieval results, respectively. The text content in the enterprise knowledge retrieval results is converted into a first feature vector, and the voice or text content in the celebrity IP knowledge retrieval results is converted into a second feature vector. Calculate the correlation score between the first feature vector and the second feature vector, and determine the first fusion weight of the first feature vector and the second fusion weight of the second feature vector based on the correlation score; Based on the first fusion weight and the second fusion weight, the first feature vector and the second feature vector are weighted and summed to generate the final feature vector, which is then used as recall enhancement information.

[0011] Furthermore, the step of inputting the intent label and recall enhancement information into a multi-objective loss function for solution to generate response content includes: A multi-objective loss function is constructed, which is obtained by weighted summation of business objective loss term, emotional objective loss term and language fluency loss term; The intent tag and recall enhancement information are used as input conditions to initialize the sequence of response content to be generated; With the goal of minimizing the multi-objective loss function, the words or tags in the response content sequence are iteratively updated until the loss function converges or a preset number of iterations is reached, thus obtaining the final response content.

[0012] Furthermore, the final response content includes: When the signal is routed to the B-end processing channel, the final response content is generated that matches the preset business scenario tone parameters; When the signal is routed to the C-end processing channel, the final response content is generated that matches the corresponding authorized vocal parameters of the star. When routing to the mixing processing channel, the final response content matching the mixed timbre parameters is generated based on the final feature vector. The mixed timbre parameters are obtained by combining the business timbre parameters and the star timbre parameters according to weights.

[0013] Furthermore, the optimization of the intent label generation process and the multi-objective loss function based on the reward signal includes: Collect the final result data of this interaction, including call duration, user key press feedback, whether a transaction was completed, and user satisfaction rating, and convert the final result data into reward points; The decision threshold used to generate intent tags is adjusted according to the reward score. The decision threshold includes a first preset threshold, a second preset threshold and their corresponding lower limit threshold. The weight coefficients of each loss term in the multi-objective loss function are updated based on the reward score; Save the adjusted decision thresholds and weighting coefficients for use in the next call interaction.

[0014] This invention provides an AI-powered mobile intelligent communication interaction method with multi-model hierarchical filtering and knowledge enhancement. It has the following beneficial effects: This invention constructs a multi-model hierarchical filtering mechanism to perform intent recognition and confidence scoring on the initial multimodal data of a call. Based on the scoring results, the call is dynamically routed to a B-end processing channel, a C-end processing channel, or a hybrid processing channel, achieving real-time triage and accurate judgment of commercial and emotional attributes. Furthermore, for different channels, it employs enterprise knowledge graph retrieval and celebrity IP knowledge vector library matching to generate recall enhancement information. The enterprise knowledge graph is pre-built based on structured data provided by the enterprise, including standard scripts, compliance rules, and user historical interaction records. The celebrity IP knowledge vector library is built based on user-authorized multimodal data, including voice feature vectors, emotional tone parameters, and common catchphrases, thus providing heterogeneous support for response content generation. The system is supported by knowledge of sexuality. By constructing a multi-objective loss function that includes business objective loss terms, emotional objective loss terms, and language fluency loss terms, the system iteratively generates response content with the goal of minimizing this loss function. This ensures that the generated content meets both the compliance and conversion rate requirements of the enterprise's business and the emotional expectations and personalized expression needs of the user. Finally, by collecting interaction result data as reward signals, the system optimizes the decision threshold for intent tag generation and the weight coefficients of the multi-objective loss function online, achieving continuous evolution and personalized adaptation of the model. In a single call, the system achieves the synergistic output of business objectives and emotional value, solving the technical problems of single objectives and value fragmentation in existing technologies, and achieving a dual improvement in business efficiency and user experience. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the steps of the AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement of the present invention. Figure 2 This is a data transmission flowchart of the AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figures 1 to 2 As shown, an AI-powered mobile intelligent communication interaction method based on multi-model hierarchical filtering and knowledge enhancement includes the following: Step 100: Acquire multimodal initial data of the call, perform feature extraction and fusion on the multimodal initial data, and output the call intent label and corresponding confidence score. Acquiring the multimodal initial data includes obtaining the caller ID, called number, and number authentication label through the system interface; acquiring the voice segment at the start of the call through the AI ​​phone's microphone and transcribing the voice segment in real time to obtain the real-time transcribed text; and obtaining the call's timestamp information. The intent label is used to identify the core purpose of the call, such as marketing promotion, after-sales follow-up, birthday wishes, holiday greetings, etc., determining the direction of subsequent processing. The confidence score reflects the reliability of the current recognition result; the higher the confidence score, the clearer the current intent; if the scores of multiple intents are close, it indicates that the call has mixed attributes.

[0018] Step 200: Based on intent tags and confidence scores, a routing decision is made to route the call to the corresponding processing channel. These channels include a B2B processing channel, a B2C processing channel, and a hybrid processing channel. The routing decision, based on the output intent tags and their confidence scores, assigns the current call to the most suitable processing channel. This ensures that subsequent knowledge retrieval and content generation accurately match the core value proposition of the call, achieving an initial classification of the call's value through routing decisions. The B2B processing channel is specifically used for calls initiated by enterprises to achieve specific business objectives, such as reaching customers, driving conversions, and providing compliance notifications. The B2C processing channel focuses on satisfying users' personalized emotional experiences, such as birthday wishes, care for family and friends, and celebrity interactions. The hybrid processing channel integrates both types of demands, generating response content that contains both commercial information and emotional warmth.

[0019] Step 300 utilizes the enterprise knowledge graph and celebrity IP knowledge vector library within the post-routing processing channel, combined with intent tags, for retrieval and matching to generate recall enhancement information. The enterprise knowledge graph provides the relationships between enterprise entities and various nodes, facilitating the rapid retrieval of related nodes and paths based on enterprise identifiers and current intent tags as query conditions, generating the most matching fragment combination to form recall enhancement information. The celebrity IP knowledge vector library generates corresponding vectors based on celebrity authorization data, forming a vector library. Each entry in the celebrity IP knowledge vector library is associated with a celebrity identifier and sentiment tag, enabling rapid calculation of the similarity between the query vector and each vector in the celebrity IP knowledge vector library when using intent tags as query conditions, thus selecting and determining recall enhancement information. Recall enhancement information transforms the static knowledge base into dynamically adapted data references that can be directly used to generate content, ensuring that the response content is both based on evidence and accurately matches the current intent.

[0020] Step 400 involves inputting the intent label and recall enhancement information into a pre-trained response generation model. This model is trained to minimize a multi-objective loss function. The response content is generated through a single forward propagation of the model. By solving the multi-objective loss function, a forced balance is maintained between business compliance, emotional appeal, and natural language during the generation process. This avoids content bias caused by a single dominant objective. For example, in a mixed processing channel, this process can generate a response that uses a celebrity's lively tone to remind the user to repay a loan, thus providing both commercial information and a friendly feel. The generated response content is a sequence of text or speech parameters that integrates intent, knowledge, and multi-objective constraints, and can be directly used for subsequent speech synthesis and playback.

[0021] Step 500: Output the response content and collect the final result data of this interaction as a reward signal. Based on the reward signal, optimize the intent label generation process and multi-objective loss function to complete the intelligent communication interaction. The final result data directly reflects the success level of this interaction and is an objective indicator for measuring interaction quality. For example, whether a transaction is completed reflects the achievement of business goals, and user satisfaction rating reflects the emotional experience effect. As a feedback scalar, the reward signal plays a crucial role in optimizing the intent label generation process. If the reward signal is high, it indicates that the current routing decision and intent label recognition are effective, and the threshold can be maintained or slightly tightened to increase recognition accuracy. If the reward signal is low, the threshold boundary needs to be relaxed, allowing more calls to enter the mixed channel or the tendency to readjust intent classification. In this way, the judgment criteria can be adaptively optimized according to the actual interaction results, making the routing more accurate. When optimizing a multi-objective loss function, the main focus is on adjusting the weight coefficients of each loss term. If the reward signal indicates that the business objective has been achieved but the emotional experience is poor (e.g., a user complains with a harsh tone), the weight of the emotional objective loss term is appropriately increased to make subsequent generation pay more attention to emotional expression. Conversely, if the emotional experience is good but the business objective has not been achieved (e.g., a user only chats but does not pay), the weight of the business objective loss term is increased. Through this online adjustment, the loss function can dynamically adapt to the preferences of different users and scenarios, achieving personalized content generation optimization. Continuous evolution and updates through the optimization process make the entire intelligent communication interaction more complete, with a better user experience and more closely aligned with the intended intelligence.

[0022] In this embodiment, the detailed implementation steps for feature extraction and fusion of multimodal initial data to output the intent label of the call and the corresponding confidence score include: Step 101: Perform feature extraction on multimodal initial data to obtain a number feature vector, a speech feature vector, and a text feature vector. The feature extraction of the multimodal initial data is a parallel and mutually independent process, which is used to extract structured information that can quantitatively characterize call attributes from raw data of different dimensions. The number feature vector reflects the identity attributes of the calling party and the called party, such as enterprise numbers and personal numbers; when extracting the number feature vector, the calling number, called number and number authentication label obtained from the multimodal initial data are parsed and mapped to a preset number type code, numbers authenticated as having enterprise attributes such as "bank" and "insurance company" are mapped to the enterprise number code "1", and unauthenticated or personal numbers are mapped to the personal number code "0", so as to form a number feature vector representing the identity attribute of the number. The speech feature vector includes information such as speech speed, tone, and emotional tendency; for the extraction of the speech feature vector, the speech segment of the first 5 seconds after the call is connected is intercepted, the segment is subjected to framing processing, then the short-term energy of each frame is calculated to reflect the volume, the fundamental frequency is calculated to characterize the pitch, and the speech speed (i.e., the number of words per unit time) is calculated to quantify the speaking rhythm. These frame-level features are statistically processed in the time dimension, the average value and variance of the tone and the average value of the speech speed in the entire segment are calculated, and finally a speech feature vector that can reflect the speed of speech, the fluctuation of pitch and the overall energy intensity is generated. The text feature vector extracts keywords and syntactic structures through semantic analysis, which is mainly analyzed and extracted by calculating the TF-IDF value. The calculation of the TF-IDF value can be directly used by accessing existing search engines and retrieval engines. The TF-IDF model is widely used in existing search engines and retrieval engines and can be directly used through computer software; for the extraction of the text feature vector, real-time automatic speech recognition is performed on the same speech segment to obtain real-time transcribed text, then the text is segmented, and stop words such as "de" and "le" are removed, then keywords are extracted by calculating the term frequency-inverse document frequency (TF-IDF) value, for example, if words such as "discount" and "activity" appear, their TF-IDF values will be relatively high. At the same time, syntactic structures such as interrogative sentences and imperative sentences in the text are identified through a preset syntactic rule base, for example, if "Do you need to handle this?" is detected, it is marked as a sales sentence pattern. Finally, the extracted keywords and their TF-IDF values, and the syntactic structure type codes are combined to form a text feature vector that characterizes the core semantics of the text.

[0023] Step 102 involves fusing the number feature vector, speech feature vector, and text feature vector using multimodal features to obtain a fused feature vector. This fused feature vector reflects the comprehensive intent tendency of the current call across three dimensions: number identity, speech prosody, and text semantics. By associating and integrating information from different modalities, a joint feature representation that more comprehensively and robustly expresses the core purpose of the call is formed, providing a decision basis for subsequent intent classification. The generation process involves parallel extraction of the number feature vector F with dimensions m, n, and p in step S101. num Speech feature vector F voice and text feature vector F text Since the dimensions and distributions of these three feature vectors may differ, they are first normalized. By normalizing the maximum and minimum values, their numerical ranges are uniformly mapped to the interval [0, 1]. Then, feature concatenation is used to connect these three normalized feature vectors in a fixed order to form a one-dimensional fusion feature vector F with dimension (m+n+p). fusion Its formal expression is: ; Where ⊕ represents the vector concatenation operation; , , These represent the normalized number feature vector, speech feature vector, and text feature vector, respectively, and the concatenated vector. All original feature information was fully preserved, providing a unified input for subsequent intent classification calculations.

[0024] Step 103: Intent classification calculation is performed based on the fused feature vector to obtain the initial score corresponding to each preset intent category. The intent classification calculation maps the fused feature vector generated in step S102 to the preset intent categories, thereby quantitatively determining which one or more preset core purposes the current call is most likely to belong to, providing a quantitative basis for the initial judgment of the call's value orientation. The initial score is calculated based on the similarity between the fused feature vector and the preset intent category; firstly, a center vector C is pre-constructed for each preset intent category i. i This center vector is obtained by averaging the fusion feature vectors manually labeled with this intent from a large number of historical calls; then, for the fusion feature vector F of the current call... fusion Calculate its relationship with the center vector C of each category. i The cosine similarity between two vectors measures their similarity by calculating the cosine of the angle between them. The value range is [-1, 1]. A larger value indicates a more consistent vector direction, meaning the current call is more similar to the intent category. The formula is: ; Among them, S i F represents the initial score of the current call relative to intent category i; fusion It is the fused feature vector of the current call; C i It is the center vector of the predefined intention category i; Represents the dot product of two vectors; and Let S represent the magnitudes (lengths) of the two vectors respectively, and calculate all the S values. i This constitutes the initial score set for the current call across various preset intent categories.

[0025] Step 104: Normalize the initial scores to obtain the confidence scores for each preset intent category. The confidence scores are used to reflect the reliability of each preset intent category. The Softmax function is used to normalize the initial score set {S1, S2, ..., S...}. n The values ​​are normalized, where n is the total number of preset intent categories; the Softmax function maps a set of arbitrary real numbers to a series of values ​​in the range [0, 1], and the sum of these values ​​is 1, which conforms to the characteristics of a probability distribution. Its calculation formula is as follows: ; Among them, P i S represents the confidence score that the current call intent belongs to the i-th preset intent category. The higher the value, the higher the confidence that the recognition result belongs to that intent category; i This is the initial score of the current call in the i-th preset intent category, calculated in step S103. This value may be positive or negative, and the range is not fixed; e is a natural constant, approximately equal to 2.71828, used as the base of the exponential function to ensure that all mapped scores are positive; the denominator is... S represents the initial score S for all n preset intent categories. j The summation after exponential operation is used to normalize the sum of confidence scores for all categories to 1, so that P... i They are comparable.

[0026] Through the above calculations, the initial score for each preset intent category is transformed into a probability distribution with a sum of 1, thus obtaining the respective confidence score P. i For example, the calculation results show that the confidence score for "marketing promotion" is 0.8, "after-sales follow-up" is 0.15, and the rest are 0.05, clearly indicating that the current call is very likely a marketing promotion call.

[0027] Step 105: Based on the confidence score, select the corresponding intent label as output from the preset intent categories. The preset intent categories include a business intent sub-label set, an affective intent sub-label set, and a mixed intent label. The confidence score includes a business intent confidence score, an affective intent confidence score, and a mixed intent confidence score. The business intent sub-label set includes marketing promotion, after-sales follow-up, notification reminders, bill collection, event invitations, satisfaction surveys, etc. These categories are usually related to one-way information transmission or business transactions between enterprises and customers. The affective intent sub-label set includes birthday wishes, holiday greetings, casual conversation with relatives and friends, schedule reminders, care and companionship, etc. These categories focus on interpersonal emotional communication and personalized care. The mixed intent label is used to indicate situations where the characteristics of both business intent and affective intent are significant or neither is significant. The intent label and its corresponding confidence score are presented in the form of key-value pairs, which together constitute the input instructions for routing decisions. The intent label is a text string that directly indicates the core purpose category of the call; the confidence score is a numerical value closely related to the label. The intent label determines the basic direction of subsequent processing and the scope of knowledge retrieval, while the confidence score is used to measure the certainty of the judgment.

[0028] In this embodiment, the detailed implementation steps for routing calls to the corresponding processing channels based on intent tags and confidence scores include: When the confidence score of business intent is higher than the first preset threshold and the confidence score of sentiment intent is lower than the second preset threshold, the call will be routed to the B-end processing channel. When the confidence score of emotional intent is higher than the second preset threshold and the confidence score of business intent is lower than the first preset threshold, the call will be routed to the C-end processing channel. When both the confidence scores for business intent and sentiment intent are higher than their respective lower thresholds, or both are lower than the first and second preset thresholds, the call will be routed to the hybrid processing channel.

[0029] The first preset threshold, the second preset threshold, and the corresponding lower threshold are determined based on statistical analysis of a large amount of historical call data and in conjunction with specific business needs. Historical calls are manually labeled (e.g., purely business-related, purely emotional, or a mix), and the confidence score distribution of business-related and emotional-related intentions is statistically analyzed to find the data quantiles that best distinguish these three types of calls, serving as the initial reference for threshold setting. The first preset threshold is used for high-confidence judgment of business-related intentions. This value is usually set relatively high, such as 0.7, to ensure that calls entering the B-end processing channel have clear business attributes and avoid mistakenly disqualifying emotionally charged communications. The second preset threshold is used for high-confidence judgment of emotional-related intentions. This value is also set relatively high, such as 0.7, to ensure that calls processed by the C-end processing channel are purely emotional interactions. The lower threshold is an auxiliary threshold used to identify mixed attributes. Its function is reflected in two scenarios: First, when both the business intent confidence score and the sentiment intent confidence score are higher than their respective lower thresholds (e.g., both greater than 0.3), it indicates that both features are significant, and the intent is mixed. Second, when both the business intent confidence score and the sentiment intent confidence score are lower than the first and second preset thresholds, it indicates that both have low confidence and the intent is unclear. Both of these situations will be routed to the mixed processing channel. This value is usually set low, such as 0.3, to capture calls that simultaneously possess both value propositions. By setting the threshold, fine-grained dynamic routing is achieved, laying the foundation for subsequent targeted content generation.

[0030] In this embodiment, the detailed implementation steps for generating enhanced recall information by utilizing the enterprise knowledge graph and celebrity IP knowledge vector library within the routing processing channel, combined with intent tags, for retrieval and matching include: Step 301: When the call is routed to the B-end processing channel, obtain the enterprise identifier associated with the call. The enterprise identifier is determined based on the number authentication tag of the calling number or the called number. Based on enterprise identifiers, the corresponding enterprise knowledge graph is retrieved. This knowledge graph, centered on enterprise entities, constructs a graph structure including standard script nodes, product knowledge nodes, user profile nodes, compliance rule nodes, and relationships. The enterprise knowledge graph generation process first collects raw data from internal systems such as the enterprise customer relationship management system, enterprise resource planning system, call center script library, product manuals, and compliance documents. Then, data cleaning is performed, including removing duplicate records, correcting errors, and filling in missing fields to ensure accuracy and consistency. From the cleaned data, different types of entities are extracted through rule-based matching (e.g., regular expressions matching phone numbers) and dictionary matching (e.g., matching product name dictionaries). For example, "XX Technology Co., Ltd." is identified as an "enterprise entity"; "Mr. Wang" and "Ms. Li" are identified as "user entities"; "Bill Reminder Script v3" and "Birthday Wishes Template" are identified as "standard script node entities"; and "XX Financial Products" and "XX Insurance Plan" are identified as "product knowledge entities." Secondly, semantic relationships between entities are defined, and these relationships are extracted from the data. For example, the "caller / called" relationship between "enterprise entities" and "user entities" is extracted from call records; the "must comply" relationship between "product knowledge entities" and "compliance rule entities" is extracted from product documents; and the "used for expression" relationship between "standard script node entities" and "intent tags" is defined from business rules. Then, detailed attribute information is populated for each entity. For example, the attributes of a "user entity" include "user ID," "historical consumption amount," and "last interaction time"; the attributes of a "product knowledge entity" include "product name," "annualized rate of return," and "risk level"; and the attributes of a "compliance rule entity" include "rule ID," "list of prohibited terms," ​​and "mandatory declarations." Finally, the extracted entities are used as nodes in a graph, the relationships between entities are used as edges connecting the nodes, and the node attributes are attached as key-value pairs to the nodes, all stored in a graph database to form a knowledge graph structure that can be efficiently queried.

[0031] The enterprise entity node, as the core of the enterprise knowledge graph, represents the enterprise that initiates or receives the call. It is used to associate all the enterprise's business knowledge, users, and rules, and serves as the starting point for retrieval. The standard dialogue node stores standard expressions or dialogue templates for different business scenarios (such as marketing, follow-up calls, and payment reminders), ensuring that the generated response content conforms to enterprise specifications and business requirements. The product knowledge node contains detailed information about the specific products or services offered by the enterprise (such as name, features, price, and promotional activities), used to provide accurate product recommendations and introductions in the response. The user profile node represents the call recipient and stores the user's personal information, historical consumption records, preference tags, and historical interaction feedback, used to achieve personalized responses, such as accurately addressing the user or mentioning products the user is interested in during the dialogue. The compliance rule node stores the laws, regulations, and internal norms that the enterprise must comply with in communications, such as "no guarantee of principal and return" and "risk disclosure is mandatory," used to constrain the generated content and avoid compliance risks.

[0032] Using enterprise identifiers and intent tags as query criteria, the system retrieves nodes and paths associated with the current intent tag in the enterprise knowledge graph to obtain candidate knowledge fragments. The generation process for candidate knowledge fragments involves first obtaining the associated enterprise identifier based on the current call determined during the routing decision in step S200, and locating the enterprise entity node matching that identifier in the enterprise knowledge graph as the starting point for this query; secondly, parsing the intent tag output in step S100, such as "bill payment reminder," and converting this intent tag into matching conditions in the query language of the enterprise knowledge graph; then, starting from the enterprise entity node, traversing along the preset association path to retrieve all nodes related to the intent tag. Taking the intent tag "bill reminder" as an example, a typical query path is (Enterprise Entity) - [Contains] -> (Standard Script Nodes). Then, it filters out standard script nodes containing the applicable intent tag in the attributes. Simultaneously, along another path (Enterprise Entity) - [Owns] -> (Product Knowledge Nodes), it retrieves information about products related to the "bill reminder" object (such as a credit card). Then, along (Enterprise Entity) - [Services] -> (User Profile Nodes), it retrieves the user profile nodes of the current caller. Finally, all the retrieved relevant nodes and their attribute data are used as a preliminary set of candidate knowledge fragments.

[0033] Candidate knowledge fragments are sorted and filtered, and one or more fragments with the highest matching degree with intent tags are selected and combined to generate recall enhancement information. The recall enhancement information generated by the B-end processing channel includes at least one of compliance-related text, product recommendation content, and user historical behavior summaries. The judgment of the matching degree of intent tags is mainly based on the semantic association strength between candidate knowledge fragments and intent tags and their weight in business logic. First, for the obtained set of candidate knowledge fragments, a matching degree score is calculated for each candidate knowledge fragment, and the value is assigned according to the direct relevance between the candidate knowledge fragment and the intent tag. For example, if the "applicable intent" attribute of a standard text node is exactly equal to "bill collection", then the score is 1.0; if the attribute value contains "bill collection" but is also applicable to other intents, the score is set to 0.7. Then, adjustments are made according to the business importance of the candidate knowledge fragments. For example, the "collection compliance statement" in the compliance rule node is considered to be mandatory, and its matching degree score will be multiplied by a weight coefficient greater than 1 (such as 1.5), while the score of some product introductions will also be increased if they are highly relevant to the current user profile (such as historical debt records). Finally, the base score is multiplied by the business weight to obtain the final matching score of each candidate knowledge fragment. The fragments are sorted from high to low according to the score, and the top K fragments with the highest scores (e.g., the top 3) are selected as the final recall enhancement information.

[0034] Among the enhanced recall information, compliant scripts ensure that the generated responses do not violate any laws, regulations, or company rules, reducing legal risks; for example, a statement such as "If you have any questions, please call customer service" must be included in the collection script. Product recommendation content is used to accurately push product information that users may be interested in or need, improving marketing conversion rates; for example, recommending installment repayment products to users with overdue payments. User history behavior summaries make the response content more personalized, making users feel valued and understood.

[0035] Step 302: When routing to the C-end processing channel, the user identifier associated with the call is obtained. The user identifier is determined based on the user identity information stored locally on the AI ​​phone. The user identifier includes an encrypted unique user ID, associated with the user's phone number, device ID, and user profile tags authorized locally on the phone. The locally stored user identity information mainly comes from the user's actively set personal profile in the phone system, as well as historical interaction data authorized by the user to specific applications (such as celebrity interaction apps) during phone use. This data is stored in a secure, isolated area of ​​the phone after being anonymized with the user's authorization. If the calling or called number of the current call cannot match any user identifier stored locally, the system will determine that the user is a temporary visitor. In this case, highly personalized celebrity IP knowledge retrieval cannot be performed. As an alternative, the intent tag of the call and general emotional tags (such as "warm" and "lively") will be used as query conditions to match a pre-built, non-personalized general emotional knowledge vector library. The general emotional knowledge vector library contains general voice and text features of different emotional expression modes and can directly use the existing default large database to generate basic emotional recall enhancement information.

[0036] Based on user identifiers, the corresponding celebrity IP knowledge vector library is retrieved. This library is a multimodal vector library constructed from user-authorized celebrity voice samples, images, catchphrases, and historical interaction records between users and celebrities. The celebrity IP knowledge vector library is pre-built based on user-authorized multimodal celebrity data, which includes at least celebrity voice samples, images, commonly used catchphrases, and historical interaction records between users and celebrities. Corresponding voice feature vectors and text feature vectors are obtained through feature extraction and stored in the vector library. Each vector in the library is associated with a celebrity identifier and emotional intent tag. In the celebrity IP knowledge vector library, each knowledge entry (e.g., a celebrity voice sample or a celebrity catchphrase) is first transformed into a feature vector (e.g., voice feature vector V) through feature extraction. voice Text feature vector V text Then, in the database, this feature vector is associated with two key metadata fields: one is the celebrity identifier, and the other is the emotional intent tag (such as "warm wishes" or "humorous banter"). This association is achieved through key-value pairs or relational tables in the database. When querying, the relevant vector can be quickly filtered and located by the celebrity identifier and emotional intent tag.

[0037] Using user identifiers and intent tags as query criteria, the similarity between the query and each vector in the celebrity IP knowledge vector library is calculated. Knowledge entries corresponding to one or more vectors with similarity exceeding a preset vector threshold are selected. The similarity calculation process first retrieves the celebrity identifiers preferred by the user from the user profile based on the user identifier. Then, combined with intent tags, such as birthday wishes, all vector subsets are filtered from the celebrity IP knowledge vector library. Next, the intent tag of the current call and user preference information are combined into a query vector Q; this query vector can be a weighted sum of the vector obtained from the embedding layer of the text tag "birthday wishes" and the celebrity identifier vector. Finally, the similarity between the query vector Q and each vector V in the filtered vector library is calculated. j The cosine similarity between them is calculated using the following formula: ; Among them, Sim j V represents the similarity score between the query and the j-th knowledge item; Q is the combined query vector; V j It is the feature vector of the j-th entry in the celebrity IP knowledge vector base; It is a dot product; and The Sim is calculated by taking the moduli of the query vector and the entry vector, respectively. j The higher the value, the more applicable the knowledge item is in the current scenario.

[0038] The selected knowledge items are combined to generate recall enhancement information, which includes at least one of the following: celebrity tone parameters, emotional tone markers, commonly used catchphrases text, and background music markers. The selected knowledge items primarily consist of multimodal data of celebrities corresponding to high similarity scores, such as celebrity tone parameters (prosody and timbre features used for speech synthesis); commonly used catchphrases text; background music markers; and emotional tone markers, such as warm or funny. These items provide personalized material for subsequent response content generation, enabling AI to mimic the tone, mannerisms, and style of celebrities to interact emotionally with users, greatly enhancing the user experience. The process of selecting knowledge items involves calculating all Sim... j The scores are sorted from highest to lowest, and a similarity threshold T is set. sim (e.g., 0.75), only retain Sim j ≥T simFor each knowledge item, to ensure the richness of the generated content, it is categorized according to emotional tone markers among those that meet the threshold. The highest-scoring item from each category is selected. For example, the highest-scoring catchphrase in the "warm" category and the highest-scoring tone parameter in the "funny" category are selected. Finally, 3-5 knowledge items are determined as the candidate set for recall. When combining the content from the selected knowledge items, based on the emotional expression framework corresponding to the intent tag, the selected knowledge items are filled into the corresponding positions of the main framework. Pre-set connecting words or transition sentences are used to naturally splice together content fragments from different sources to form a coherent text, resulting in the final enhanced recall information.

[0039] Among the enhanced recall information, the celebrity tone parameter controls the speech synthesis, giving the generated speech the timbre and rhythm of a specific celebrity, achieving a human-like auditory effect. The emotional tone identifier guides the emotional tone of the generated content, ensuring that the emotional color of the response matches the call's intent. Commonly used catchphrases are used to increase the fun and recognizability of the responses, making users feel familiar and comfortable when interacting with the AI, strengthening the sense of identification with the celebrity IP. The background music identifier recommends or specifies background music that matches the current emotional atmosphere, multimodally enhancing the emotional immersion of the user experience.

[0040] Step 303: When routing to the hybrid processing channel, both enterprise knowledge graph-based retrieval and celebrity IP knowledge vector library-based retrieval are performed simultaneously to obtain enterprise knowledge retrieval results and celebrity IP knowledge retrieval results, respectively. Enterprise knowledge retrieval results include text fragments such as compliant wording, product recommendation content, and user historical behavior summaries retrieved from the enterprise knowledge graph, used to provide accurate, compliant, and business-specific factual information. Celebrity IP knowledge retrieval results include celebrity tone parameters, emotional tone identifiers, and commonly used catchphrases retrieved from the celebrity IP knowledge vector library, used to provide information on personalized and emotionally rich expressive styles.

[0041] The text content from the enterprise knowledge retrieval results is converted into a first feature vector, and the audio or text content from the celebrity IP knowledge retrieval results is converted into a second feature vector. The first feature vector is used to capture the semantic connotation of business information, representing the semantics of business knowledge in the vector space, which facilitates fusion calculation with the sentiment vector. The second feature vector represents the unique features of sentiment expression in the vector space and is used for matching and fusion with the business vector.

[0042] Calculate the relevance score between the first and second feature vectors, and determine the first fusion weight of the first feature vector and the second fusion weight of the second feature vector based on the relevance score. The relevance score R is a scalar value used to measure the semantic matching degree between the currently recalled enterprise business information and celebrity sentiment information in the mixed channel. The relevance score is a key basis for subsequently determining the fusion weights of the two. If the relevance score is high, it indicates that the business information and sentiment information are naturally semantically compatible, and the sentiment component can be given a higher weight; if the relevance score is low, it indicates that the combination of the two may be forced, and the sentiment weight needs to be carefully controlled, or even the fusion strategy adjusted. The first feature vector E is calculated... biz With the second eigenvector E emo The cosine similarity between them is used as the relevance score R, and the formula is: ; Among them, E biz It is the feature vector of the enterprise knowledge recall result; E emo It is the feature vector of the celebrity IP knowledge retrieval result; the value range of R is [-1, 1], and the larger the value, the more consistent the directions of the two in the vector space and the stronger the semantic correlation.

[0043] The determination of the first and second fusion weights is based on two factors: the intent tag set of the current call and the calculated relevance score. First, a basic weight is set based on the intent tags. (Setting the basic service weights) and basic sentiment weight Then, an adjustment factor α is calculated based on the correlation score R. α can be calculated according to a preset function. A sigmoid function is used, such that α approaches 1 when R is high and approaches 0 when R is low. The parameter k controls the sensitivity of the adjustment. Finally, the first fusion weight W is calculated. biz Second fusion weight W emo : ; ; Specifically, when the relevance is high (large α), the sentiment weight increases; when the relevance is low (small α), the business weight is maintained or increased, thus avoiding disharmonious integration. Finally, regarding W... biz and W emo Normalization is performed to obtain the normalized first fusion weight. Second fusion weight , making W biz +W emo =1.

[0044] Based on the first fusion weight and the second fusion weight, the first feature vector and the second feature vector are weighted and summed to generate the final feature vector, which is then used as recall enhancement information.

[0045] The first fusion weight after normalization Second fusion weight Then, for the first eigenvector E biz Second eigenvector E emo Perform a weighted summation to generate the final feature vector F. final ; ; Final eigenvector F final By integrating the semantics of business information and the semantics of sentiment expression, a joint representation of response content that contains both commercial information and emotional warmth is generated. The final generated recall enhancement information is the final feature vector F. final This is a highly abstract content that integrates business facts and emotional style. Subsequent responses will use this vector as the core input. Through solving a multi-objective loss function, it will be decoded into a specific response text that mixes business keywords and emotional tone words. This will guide the speech synthesis engine to output a mixed voice parameter that combines corporate customer service standards with the characteristics of celebrity voices, ultimately achieving the synergistic output of business goals and emotional value.

[0046] In this embodiment, the detailed implementation steps for generating response content by inputting intent tags and recall enhancement information into a multi-objective loss function for solution include: Step 401: Construct a multi-objective loss function. This function is obtained by weighted summation of a business objective loss term, an emotional objective loss term, and a language fluency loss term. The business objective loss term measures the degree of matching between the generated content and preset business keywords and compliance rules; the emotional objective loss term measures the consistency between the emotional tendency of the generated content and the target emotional identifier; and the language fluency loss term measures the linguistic fluency of the generated content. The multi-objective loss function is a weighted summation function used to simultaneously constrain and balance the three dimensions of business compliance, emotional appeal, and language naturalness during the response content generation process, avoiding a bias where the generated content is dominated by a single objective. Its calculation formula is as follows: ; Among them, L total This represents the total multi-objective loss value, a scalar. The smaller this value, the better the overall performance of the generated response across the three objective dimensions. This value will be used as the minimization objective in the iterative optimization of step S403; L biz (G,R) bizThe ) represents the business objective loss item, used to measure the degree of matching between the generated response content G and the business requirements. Specifically, the calculation begins with the recall enhancement information R. biz Extract a set of business keywords from the compliance script text and product keywords generated in step S301; then, calculate the coverage of these keywords in the generated response content G, as well as the semantic similarity between the generated response content and the compliance rule template; if the generated response content omits key business terms or violates compliance rules, the loss value increases. emo (G, T) emo The term ) represents the sentiment loss term, used to measure the sentiment tendency of the generated response content G relative to the expected sentiment intention label T. emo Consistency; T emo The intent tag, derived from the output of step S100, is used in the calculation by scoring the response content G with sentiment using a pre-set sentiment classifier, and then compared with T. emo The corresponding standard sentiment vector is used to calculate the difference (e.g., mean squared error). If the generated content has a stiff tone and deviates from the expected sentiment, the loss value for this item will increase. flu (G) represents the language fluency loss term, used to measure whether the generated response content G conforms to the fluency and grammatical rules of natural language; by evaluating the probability of the response content G appearing, the lower the probability, the more awkward and illogical the expression, and the larger the loss value of this term. λ biz , λ emo , λ flu Let represent the weight coefficients of the business objective loss term, the emotional objective loss term, and the language fluency loss term, respectively. These three coefficients are preset hyperparameters and satisfy λ. biz +λ emo +λ flu =1; this determines the degree of importance attached to each objective during the optimization process; for example, in the B-end processing channel, λ biz It will be set higher; in the C-end processing channel, λ emo These weighting coefficients will be dynamically adjusted in step S503 based on the reward signal, taking precedence over the previous ones.

[0047] The entire computational and iterative update process of the constructed multi-objective loss function can be directly completed using the industrial-grade mathematical optimization solver Gurobi. As a leading global mathematical optimization engine, Gurobi provides complete native support for multi-objective optimization problems. In Gurobi, multiple objective functions can be added to the model using the `setObjectiveN` function, and each objective function can have its priority and weights set independently. The three loss terms L... biz L emo L fluEach loss term can be added as an independent objective function to the Gurobi model for calculating its value. The parameter `MultObjMethod` controls how Gurobi handles multiple objectives; setting it to 0 prioritizes each objective, while setting it to 1 combines all objectives into a weighted sum. In this technical solution, the three loss terms are combined using a weighted summation method; therefore, mode 1 can be selected to combine all objectives into a single weighted sum for overall optimization.

[0048] Step 402: Initialize the response content sequence to be generated by using the intent label and recall enhancement information as input conditions. The response content sequence is the mathematical representation of the response content to be generated, and is a sequence of word units or control tags arranged in chronological order. The response content sequence consists of two parts: a text word sequence and control parameter tags attached to each word. The text word sequence provides the words, text, and other content of the response, while the control parameter tags attached to each word are used to guide the final speech synthesis, such as timbre identification, emotional tone identification, and pause duration. As the initial state of the response content generation process, the response content sequence carries the intent label output in step S100 and the recall enhancement information output in step S300, providing the starting point and basic materials for subsequent iterative optimization. The initialization process transforms abstract intent labels and recall enhancement information into concrete, computer-processable symbol sequences. The initialization process involves matching a basic response template from a pre-defined intent response template library based on the intent label output in step S100; filling the placeholders in the template with the specific content from the recall enhancement information generated in step S300; and obtaining the initial text word sequence after filling. Simultaneously, based on the class of the current processing channel, initial control parameter markers, such as default speech rate and pitch, are appended to the initial text word sequence.

[0049] Step 403: With the goal of minimizing the multi-objective loss function, the words or tags in the response content sequence are iteratively updated until the loss function converges or a preset number of iterations is reached, yielding the final response content. By iteratively updating and fine-tuning the word selection or control tags in the response content sequence, the loss value of the entire response content sequence on the multi-objective loss function defined in step S401 is gradually reduced, ultimately finding an optimal or near-optimal solution that balances business, sentiment, and language fluency. The calculation process uses gradient descent for iterative updates. First, the current response content sequence G is... t Let t represent the sequence of responses in the t-th iteration, which is input into the multi-objective loss function to calculate the current loss value. Next, calculate the loss function. Relative to the response content sequence G t The gradient of each adjustable element (such as the embedding representation of a lexical or the value of a control token) The gradient indicates the direction in which the sequence elements should be adjusted to most effectively reduce the loss. Then, following the rules of gradient descent, the elements in the sequence are updated: Where η is the learning rate, controlling the step size for each update. For example, if the gradient shows that fine-tuning the weight of a word towards "warmth" can reduce the sentiment loss term, then the sentiment control marker corresponding to that word will be adjusted accordingly in the response content sequence. Finally, the above steps are repeated until the loss function L... total Convergence (i.e., the loss value no longer decreases significantly) or reaching the preset maximum number of iterations, with each iteration generating a new, better sequence of response content.

[0050] When the iteration stopping condition is met, the latest response content sequence G is... final This is the final generated response content, which needs to be... final Transforming the mathematical representation into a practically usable output form; first, converting the final sequence G... final The text word sequence is mapped back to natural language text through a vocabulary to obtain a complete response text; then from G... final The final determined speech synthesis control parameters are extracted, including timbre identifier, emotional tone identifier, speech rate, pitch curve, etc.; finally, the response text and speech synthesis parameters are combined into the final output, which is the output result of step 400 and used for subsequent speech synthesis and playback.

[0051] The multi-objective loss function is solved based on a pre-trained response generation model. During model training, the generation model is iteratively trained with the goal of minimizing the multi-objective loss function until the loss function converges, resulting in a trained generation model. During the inference phase of real-time calls, intent tags and recall enhancement information are input into the trained generation model. The response content can be obtained through a single forward propagation calculation, eliminating the need for iterative updates during the inference phase and ensuring low latency for real-time interaction on the mobile device.

[0052] The step S403, which iteratively updates the response content sequence using gradient descent, can be directly implemented using the optimizer module (torch.optim) of the PyTorch framework. As a mainstream deep learning framework, PyTorch's optimizer module is specifically designed for gradient descent algorithms, meeting the iterative update requirements of step S403. The response content sequence G to be generated can be... t (Including the text word sequence and its attached control parameter labels) is represented as a trainable tensor in PyTorch. It is initialized using the `torch.optim.SGD` optimizer with a suitable learning rate η (e.g., 0.01). In each iteration, the current sequence G is first... tInput to the multi-objective loss function L total In the process, the ladder is calculated by calling the loss.backward() function through the automatic differentiation mechanism. Then, `optimizer.step()` is called to update the elements in the sequence according to the gradient descent rule. PyTorch's SGD optimizer also supports momentum parameter settings, which can accelerate convergence and help escape local optima. By setting parameters such as `momentum=0.9`, the efficiency and stability of iterative updates can be further improved. The entire iterative process is repeated until the loss function converges or the preset maximum number of iterations is reached, finally obtaining the optimized response content sequence G. final .

[0053] In this embodiment, the final response includes: When routed to the B-end processing channel, based on the compliance-related script text and product recommendation content in the recall enhancement information, a response text containing compliance terminology, product name, and user address is generated, and matched with preset business scenario voice parameters. These business scenario voice parameters are obtained from standard script nodes in the enterprise knowledge graph, providing the B-end processing channel with a professional and consistent brand voice image. For example, the final response generated by the B-end processing channel might be: "Dear Mr. Li, your bank credit card statement for January is overdue, with an outstanding amount of 3500 yuan. Please repay today through the official app or online banking to avoid affecting your credit record. If you have any questions, please call customer service at 95555." The voice is a professional and composed customer service tone.

[0054] When routed to the C-end processing channel, the system generates a response text containing the celebrity's common catchphrases and emotional expressions based on the celebrity tone parameters and emotional tone identifiers in the recall enhancement information, and matches the corresponding celebrity authorized voice parameters. The celebrity authorized voice parameters are obtained from the celebrity IP knowledge vector library, enabling the speech synthesis engine to mimic the voice and speaking style of a specific celebrity, providing a highly human-like emotional interaction experience. For example, the final response generated by the C-end processing channel might be: "Hey bro! Happy birthday! Wow, not bad, you're another year older today. Wishing you happiness and all your wishes come true every day! Remember to eat cake!" The voice is a specific celebrity voice, with a lively and friendly tone.

[0055] When routed to the hybrid processing channel, a response text integrating business information and emotional expression is generated based on the final feature vector. This response text contains both business keywords and emotional tone words, and is matched with hybrid timbre parameters. These parameters are obtained by weighted combination of business timbre parameters and celebrity timbre parameters. The hybrid timbre parameters are calculated to blend business professionalism with celebrity approachability, ensuring that the generated speech in the hybrid processing channel possesses both the seriousness of business information and emotional appeal. The calculation process for the hybrid timbre parameters first obtains business scenario timbre parameters from the enterprise knowledge graph, represented as a high-dimensional timbre vector V. biz ; Obtain the authorized timbre parameters of the celebrity from the celebrity IP knowledge vector database, and represent them as timbre vector V. star Next, the final feature vector F generated in step S303... final In the process, the business timbre weight W is analyzed. biz And the weight of star voice W star Satisfying W biz +W star =1, its value is determined by the correlation score calculation in step S303. Then, a weighted linear interpolation is performed on the two timbre vectors to generate the mixed timbre vector V. mixed , Finally, the mixed timbre vector V mixed The input is fed into the voice decoder of the speech synthesis engine, where it is decoded to generate the final sequence of mixed voice parameters. This sequence drives the speech synthesizer to output a mixed voice that is both professional and distinctive, reminiscent of a celebrity. For example, the final response generated by the mixed processing channel is: "Hello Mr. Li! I'm your smart assistant, Xiao Jie. Happy birthday! Just a friendly reminder that your credit card bill is due today, so please remember to pay it when it's convenient so you can enjoy your birthday in peace!" The voice blends the professional tone of a customer service representative with the lively intonation of a celebrity.

[0056] The timbre parameters are timbre feature vectors recognizable by the speech synthesis engine, including dimensions such as fundamental frequency mean, formant parameters, speech rate coefficient, and timbre embedding vector. The business scenario timbre parameters are timbre vectors obtained by feature extraction from pre-recorded standard customer service speech, stored in the standard dialogue nodes of the enterprise knowledge graph. The celebrity-authorized timbre parameters are timbre vectors obtained by feature extraction from celebrity-authorized speech samples, stored in the celebrity IP knowledge vector library. The mixed timbre parameters are obtained by weighted linear interpolation of the business timbre vector and the celebrity timbre vector. Inputting the mixed timbre vector into the timbre decoder of the speech synthesis engine generates synthesized speech with corresponding timbre features.

[0057] In this embodiment, the detailed implementation steps for optimizing the intent label generation process and the multi-objective loss function based on the reward signal include: Step 501: Collect the final result data of this interaction. This data includes call duration, user key press feedback, whether a transaction was completed, and user satisfaction rating. The final result data is then converted into reward points. This data is collected in real-time through the AI ​​phone's system interface and user interface. Call duration is automatically recorded by a system timer, reflecting the depth of interaction between the user and AI. Too short a duration may indicate the user hung up (negative); a moderate duration indicates normal communication; too long a duration may indicate unresolved issues. User key press feedback is collected through the interactive voice response system's key press capture module, directly reflecting the user's immediate reaction to the interaction, such as confirming receipt of information (positive), requesting to be transferred to a human (neutral), or filing a complaint (negative). Whether a transaction was completed is obtained through integration with the enterprise's backend order system or customer relationship management system. For B2B processing channels, this is a core business indicator, directly measuring whether business goals have been achieved, such as whether repayment was completed or products were purchased. User satisfaction rating is collected through SMS messages or app-push surveys after the call ends. For C2C processing channels, this is a core experience indicator for emotional interaction, directly reflecting the user's satisfaction with the emotional interaction effect. By collecting the final result data to form objective and quantifiable interaction quality evaluation indicators, we can provide real-world feedback signals for subsequent model optimization, enabling the entire intelligent communication and interaction system to continuously learn and evolve.

[0058] The reward score D is a scalar value, typically ranging from [0, 1] to [-1, 1], used to quantify the overall success of the interaction. A high reward score indicates a very successful interaction, and its routing decisions and content generation strategies will be strengthened; a low reward score indicates a failed interaction, and its related strategies will be adjusted. The reward score D is obtained by weighting and summing the various indicators in the final result data according to business objectives. The calculation formula is as follows: ; Among them, I deal This is an indicator variable representing whether a transaction has been completed; 1 indicates a completed transaction, and 0 indicates otherwise. S sat This represents the user satisfaction rating, normalized to the interval [0, 1]. inter This represents a normalized value indicating call duration, such as normalized based on industry average duration. Calls that are too long or too short will decrease this score. F feedback This represents the rating mapping for user button feedback, such as a satisfaction rating of 1 and a complaint rating of -0.5. α, β, γ, and δ represent the weighting coefficients of each indicator, preset according to business priority, and α+β+γ+δ=1.

[0059] Step 502: Adjust the decision threshold used when generating intent labels based on the reward score. The decision threshold includes a first preset threshold, a second preset threshold, and their corresponding lower limits. The first preset threshold T in the routing decision is dynamically and adaptively adjusted based on the reward score D. business Second preset threshold T emotion and lower limit threshold T lower The calculation and adjustment process is as follows: First, calculate the threshold adjustment amount. This is based on the current reward score D and the historical average reward score. The difference is used to calculate the threshold adjustment amount ΔD; ; Where μ is the adjustment step size coefficient (default is 0.05), which controls the magnitude of each adjustment. When When this occurs, it indicates that the interaction is better than average, and the current routing strategy should be strengthened to move the threshold towards greater precision, i.e., increase the threshold corresponding to high confidence scores; when If the interaction is below average, the routing strategy should be relaxed to allow more calls to enter the hybrid channel in order to explore better handling methods.

[0060] Secondly, adjust the first preset threshold T business Second preset threshold T emotion : ; ; Where, σ biz and σ emo This is the sensitivity coefficient of the first and second preset thresholds, set according to business and emotional needs respectively, and is usually 1 for both. When ΔD is positive, the two high confidence thresholds are increased, meaning that the requirements for entering the B-end processing channel or C-end processing channel are more stringent, and only calls with very high confidence scores can enter, which helps to improve the accuracy of processing within the channel; when ΔD is negative, the thresholds are decreased, allowing more calls to enter the dedicated channel for exploration.

[0061] Then, adjust the lower limit threshold T. lower , ; Where, σ lower It is a sensitivity coefficient of the lower threshold. When ΔD is positive (high reward), the lower threshold decreases, which means that the scope of judgment for ambiguous intent is narrowed and more calls will be explicitly routed to dedicated channels. When ΔD is negative (low reward), the lower threshold increases, which means that the scope of judgment for ambiguous intent is expanded, causing more calls to enter the hybrid channel in an attempt to improve the experience through fusion generation.

[0062] Finally, a boundary check is performed on the adjusted threshold to ensure it is within a reasonable range. For example, limiting T... business and T emotion Between [0.5, 0.9], T lower Between [0.1, 0.5], avoid excessive threshold adjustment that could cause the routing policy to fail.

[0063] Step 503: Update the weight coefficients of each loss term in the multi-objective loss function according to the reward score D. biz , λ emo and λ flu It is a feedback-based online learning process, and the update process includes: First, analyze the final result data of this interaction, analyze the achievement of business goals (such as whether a transaction was completed) and the effect of emotional experience (such as user satisfaction rating), and determine whether the contribution of business goals is insufficient, the contribution of emotional goals is insufficient, or both.

[0064] Secondly, based on the reward score D and the preset target reward score D... target Based on the difference and the above analysis results, the adjustment δ of the weight coefficient of each loss item is calculated. biz δ emo δ flu For example, if D < D target Furthermore, if the analysis shows that business objectives were not achieved (e.g., no transaction was completed) but the emotional experience was positive (e.g., high satisfaction), then the constraints on business objectives should be strengthened, i.e., the λ value should be increased. biz Decrease λ emo The adjustment amount is calculated using the following formula: ; ; ; Where v is the weight adjustment step size (default is 0.1); I deal This is a transaction indicator variable. The formula above indicates that when the reward is insufficient and no transaction is completed, the business weight increases; the weights of sentiment and fluency decrease proportionally to keep the total weight of 1.

[0065] Then, update the weight coefficients: ; ; ; Finally, ensure that the sum of the updated weight coefficients is 1, and that each weight is within a preset range (e.g., each λ≥0.1). If the weights exceed the range, then truncate and renormalize them.

[0066] By adjusting online, the weights of the multi-objective loss function can dynamically adapt to the needs of different business scenarios and user preferences based on real-time feedback. For example, during peak marketing seasons, the weight of business objectives can be automatically increased to pursue higher conversion rates; in customer care scenarios, the weight of emotional objectives can be automatically increased to improve user satisfaction and loyalty. Since different user groups have varying sensitivities to business and emotional aspects, continuous weight adjustments allow the system to learn and remember the preferences of specific users or user groups. In subsequent interactions, a more suitable weight configuration can be adopted, achieving personalized content generation optimization. The dynamic update mechanism avoids a single objective dominating the generation strategy in the long term, ensuring that the system continuously finds the optimal balance between business objectives, emotional objectives, and language naturalness, achieving a dual improvement in business efficiency and user experience.

[0067] Step 504: Save the adjusted decision threshold and weight coefficients for use in the next call interaction.

[0068] This invention constructs a multi-model hierarchical filtering mechanism to perform intent recognition and confidence scoring on the multimodal initial data of a call. Based on the scoring results, the call is dynamically routed to a B-end processing channel, a C-end processing channel, or a hybrid processing channel, achieving real-time triage and accurate judgment of commercial and emotional attributes. Furthermore, for different channels, it uses enterprise knowledge graph retrieval and celebrity IP knowledge vector library matching to generate recall enhancement information. The enterprise knowledge graph is pre-built based on structured data provided by the enterprise, including standard scripts, compliance rules, and user historical interaction records. The celebrity IP knowledge vector library is built based on user-authorized multimodal data, including voice feature vectors, emotional tone parameters, and common catchphrases, thereby providing support for response content generation. It provides heterogeneous knowledge support; by constructing a multi-objective loss function that includes business objective loss terms, emotional objective loss terms, and language fluency loss terms, it iteratively generates response content with the goal of minimizing this loss function. This ensures that the generated content not only meets the compliance and conversion rate requirements of the enterprise's business, but also meets the emotional expectations and personalized expression needs of the user. Finally, by collecting interaction result data as reward signals, the decision threshold for intent tag generation and the weight coefficients of the multi-objective loss function are optimized online, realizing the continuous evolution and personalized adaptation of the model. In a single call, it achieves the synergistic output of business objectives and emotional value, solving the technical problems of single objectives and value fragmentation in existing technologies, and achieving the dual improvement of business efficiency and user experience.

[0069] This application also provides an electronic device. The electronic device may include one or more processors and one or more memories. The memories store computer-readable code, which, when executed by the one or more processors, can perform the AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical filtering and knowledge enhancement as described above.

[0070] The method according to the embodiments of this application can also be implemented using the architecture of the electronic device shown in this application. The electronic device may include a bus, one or more CPUs, ROM, RAM, a communication port connected to a network, input / output, a hard disk, etc. The storage device in the electronic device, such as a ROM or hard disk, may store the AI ​​mobile phone intelligent communication interaction method for multi-model hierarchical filtering and knowledge enhancement provided in this application. Furthermore, the electronic device may also include a user interface. Of course, the architecture shown in this application is merely exemplary; when implementing different devices, one or more components in the electronic device shown in this application may be omitted according to actual needs.

[0071] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a reference structure" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0072] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An AI-powered mobile intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement, characterized in that, The method includes: Acquire multimodal initial data of the call, extract and fuse features from the multimodal initial data, and output the intent label of the call and the corresponding confidence score; Routing decisions are made based on the intent tags and confidence scores to route calls to corresponding processing channels. These processing channels include B-end processing channels, C-end processing channels, and hybrid processing channels. The confidence scores include business intent confidence scores, emotional intent confidence scores, and hybrid intent confidence scores. When the confidence score of business intent is higher than the first preset threshold and the confidence score of sentiment intent is lower than the second preset threshold, the call will be routed to the B-end processing channel. When the confidence score of emotional intent is higher than the second preset threshold and the confidence score of business intent is lower than the first preset threshold, the call will be routed to the C-end processing channel. When the confidence scores of business intent and sentiment intent are both higher than their respective lower thresholds, or both are lower than the first and second preset thresholds, the call will be routed to the hybrid processing channel. By utilizing the enterprise knowledge graph and celebrity IP knowledge vector library within the routing processing channel, combined with intent tags, retrieval and matching are performed to generate recall enhancement information; the B-end processing channel retrieves the enterprise knowledge graph, the C-end processing channel retrieves the celebrity IP knowledge vector library, and the hybrid processing channel retrieves both simultaneously and performs feature weighted fusion. The intent label and recall enhancement information are input into a pre-trained response generation model, which is trained with the goal of minimizing a multi-objective loss function. The response content is generated through a forward propagation of the response generation model. The response content is output, and the final result data of this interaction is collected as a reward signal. Based on the reward signal, the intent label generation process and multi-objective loss function are optimized to complete the intelligent communication interaction.

2. The AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to claim 1, characterized in that, The acquisition of multimodal initial data for the call includes: Obtain the calling number, the called number, and the number authentication label; The system acquires the audio segment at the start of the call and performs real-time transcription of the audio segment to obtain the real-time transcribed text. Get the timestamp information of the call.

3. The AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to claim 1, characterized in that, The process of extracting and fusing features from the initial multimodal data to output the intent label and corresponding confidence score of the call includes: Feature extraction is performed on the multimodal initial data to obtain number feature vector, voice feature vector and text feature vector; The number feature vector, voice feature vector, and text feature vector are fused using multimodal feature fusion to obtain a fused feature vector; Based on the fused feature vector, intent classification calculation is performed to obtain the initial score corresponding to each preset intent category; The initial scores are normalized to obtain confidence scores for each preset intent category. The preset intent categories include business intent sub-tag sets, sentiment intent sub-tag sets, and hybrid intent tags.

4. The AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to claim 1, characterized in that, The process utilizes the enterprise knowledge graph and celebrity IP knowledge vector library within the routing processing channel, combined with intent tags, for retrieval and matching to generate enhanced recall information, including: When routing to the B-end processing channel, obtain the enterprise identifier associated with the call; Based on the enterprise identifier, the corresponding enterprise knowledge graph is read. The enterprise knowledge graph is constructed with the enterprise entity as the central node, and includes a graph structure including standard language nodes, product knowledge nodes, user profile nodes, compliance rule nodes and related relationships. Using the enterprise identifier and intent tag as query conditions, retrieve the nodes and paths associated with the current intent tag in the enterprise knowledge graph to obtain candidate knowledge fragments; The candidate knowledge fragments are sorted and filtered, and one or more fragments with the highest matching degree with the intent tag are selected and combined to generate recall enhancement information.

5. The AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to claim 4, characterized in that, The process utilizes the enterprise knowledge graph and celebrity IP knowledge vector library within the routing processing channel, combined with intent tags, for retrieval and matching to generate enhanced recall information, including: When routing to the C-end processing channel, obtain the user identifier associated with the call; Based on the user identifier, the corresponding celebrity IP knowledge vector library is read. The celebrity IP knowledge vector library is a multimodal vector library constructed based on user-authorized celebrity voice samples, image pictures, catchphrases, and historical interaction records between users and celebrities. Using the user identifier and intent tag as query conditions, calculate the similarity between the query and each vector in the celebrity IP knowledge vector library, and select the knowledge entries corresponding to one or more vectors whose similarity exceeds a preset vector threshold. The content from the selected knowledge items is combined to generate recall enhancement information.

6. The AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to claim 5, characterized in that, The process utilizes the enterprise knowledge graph and celebrity IP knowledge vector library within the routing processing channel, combined with intent tags, for retrieval and matching to generate enhanced recall information, including: When routed to the hybrid processing channel, both enterprise knowledge graph-based retrieval and celebrity IP knowledge vector database-based retrieval are performed simultaneously to obtain enterprise knowledge retrieval results and celebrity IP knowledge retrieval results, respectively. The text content in the enterprise knowledge retrieval results is converted into a first feature vector, and the voice or text content in the celebrity IP knowledge retrieval results is converted into a second feature vector. Calculate the correlation score between the first feature vector and the second feature vector, and determine the first fusion weight of the first feature vector and the second fusion weight of the second feature vector based on the correlation score; Based on the first fusion weight and the second fusion weight, the first feature vector and the second feature vector are weighted and summed to generate the final feature vector, which is then used as recall enhancement information.

7. The AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to claim 6, characterized in that, The step of inputting the intent label and recall enhancement information into the pre-trained response generation model includes: A multi-objective loss function is constructed, which is obtained by weighted summation of business objective loss term, emotional objective loss term and language fluency loss term; The intent tag and recall enhancement information are used as input conditions to initialize the sequence of response content to be generated; With the goal of minimizing the multi-objective loss function, the words or tags in the response content sequence are iteratively updated until the loss function converges or a preset number of iterations is reached, thus obtaining the final response content.

8. The AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to claim 7, characterized in that, The final response content includes: When the signal is routed to the B-end processing channel, the final response content is generated that matches the preset business scenario tone parameters; When the signal is routed to the C-end processing channel, the final response content is generated that matches the corresponding authorized vocal parameters of the star. When routing to the mixing processing channel, the final response content matching the mixed timbre parameters is generated based on the final feature vector. The mixed timbre parameters are obtained by combining the business timbre parameters and the star timbre parameters according to weights.

9. The AI ​​mobile phone intelligent communication interaction method with multi-model hierarchical screening and knowledge enhancement according to claim 1, characterized in that, The optimization of the intent label generation process and multi-objective loss function based on the reward signal includes: Collect the final result data of this interaction, including call duration, user key press feedback, whether a transaction was completed, and user satisfaction rating, and convert the final result data into reward points; The decision threshold used to generate intent tags is adjusted according to the reward score. The decision threshold includes a first preset threshold, a second preset threshold and their corresponding lower limit threshold. The weight coefficients of each loss term in the multi-objective loss function are updated based on the reward score; Save the adjusted decision thresholds and weighting coefficients for use in the next call interaction.

Images