LLM-based client intention identification and response system, method and device, and medium

Abstract

The invention discloses an LLM-based customer intention recognition and response system, method, device and medium, and relates to the technical field of intelligent dialogues, the system comprises a multi-modal input processing unit used for collecting and fusing multi-source interaction data of an intelligent voice outbound system to form multi-dimensional input representation; the dynamic context enhancement LLM unit is used for performing context understanding based on the multi-dimensional input representation and a preset knowledge base and generating a user feature portrait; the reinforcement learning dialogue strategy generation unit is used for dynamically generating a dialogue strategy according to the user feature portrait and generating response content based on the dialogue strategy; and the multi-modal output unit is used for converting the response content into multi-modal output information and performing adaptive adjustment on the multi-modal output information according to the real-time scene parameters. According to the invention, accurate identification and intelligent response to the intention of the user are realized, and the interaction experience and service quality of the intelligent dialogue system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent dialogue technology, and in particular to a customer intention recognition and response system, method, device and medium based on LLM. Background Art

[0002] With the rapid development of large language model (LLM) technology, its application in scenarios such as intelligent customer service and voice outbound calls is becoming more and more extensive. Existing technologies usually use single-modal speech recognition or text processing to understand user intentions and generate response content based on preset dialogue templates. Some systems have introduced emotion recognition modules, attempting to identify the user's emotional state by analyzing the user's voice features, thereby optimizing the dialogue strategy. However, these systems often show limitations when dealing with actual dialogue scenarios: first, single-modal feature extraction cannot fully capture the user's interaction state, resulting in inaccurate understanding of intentions; second, the response generation method based on fixed templates lacks flexibility and is difficult to adapt to complex and changing dialogue scenarios; third, existing systems generally use independent feature processing modules, and there is a lack of effective information interaction and collaborative optimization mechanisms between modules, resulting in less than ideal overall system performance.

[0003] More importantly, existing technologies have serious problems with context understanding bias and attention tracking failure when dealing with long conversations. Traditional methods usually use a context memory mechanism with a fixed window size, which cannot dynamically adjust the information retention strategy according to the user's cognitive state. At the same time, the generation process of dialogue strategies often ignores the dynamic changes of user attention, emotional state, and topic preferences, resulting in a significant deviation between system responses and actual user needs. In addition, when performing multimodal output, existing systems rarely consider the impact of real-time scene parameters, which greatly reduces the naturalness of system responses and interactive experience. These problems are particularly prominent in scenarios that require highly personalized and precise services, such as insurance product recommendations.

[0004] Therefore, how to effectively improve the performance of intelligent dialogue systems in actual application scenarios, especially how to accurately identify and promptly respond to changes in users' cognitive states, has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above-mentioned problems, the present invention provides a customer intent identification and response system, method, device and medium based on LLM to solve the problems mentioned in the background technology.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a customer intention recognition and response system based on LLM, comprising: a multimodal input processing unit, used to collect and fuse multi-source interaction data of an intelligent voice outbound call system to form a multi-dimensional input representation; a dynamic context enhanced LLM unit, which performs context understanding based on the multi-dimensional input representation and a preset knowledge base to generate a user feature portrait; a reinforcement learning dialogue strategy generation unit, used to dynamically generate a dialogue strategy based on the user feature portrait, and generate response content based on the dialogue strategy; a multimodal output unit, used to convert the response content into multimodal output information, and adaptively adjust the multimodal output information according to real-time scene parameters.

[0007] As a preferred solution of the LLM-based customer intention recognition and response system described in the present invention, the multimodal input processing unit obtains multi-source interaction data in the intelligent voice outbound call system in real time through a multi-channel data acquisition module, specifically: the acoustic features of the customer's voice are obtained through the audio acquisition channel; the customer's voice is transcribed into text information in real time through the voice recognition engine, and the linguistic features in the text are extracted; the rhythmic features in the voice are extracted through the emotion recognition module; and the background noise features during the call are obtained through the environmental acoustic analysis unit.

[0008] As a preferred solution of the LLM-based customer intention recognition and response system described in the present invention, the multimodal input processing unit adopts a multi-level feature fusion strategy to fuse the acquired multi-source interaction data, specifically: normalizing the semantic features, linguistic features, prosodic features, and background noise features; weighting the importance of different modal features; mapping the weighted features to a unified feature space through a multimodal feature encoder to generate a feature representation aligned between modalities; and adopting a cross-modal fusion network to organically integrate the features of each modality through the complementarity and synergy between modalities to form a multi-dimensional input representation.

[0009] As a preferred solution of the LLM-based customer intention recognition and response system described in the present invention, the process of generating the user feature portrait includes the following steps: the dynamic context enhanced LLM unit obtains the multi-dimensional input representation and inputs it into the pre-trained feature conversion network to obtain an initial feature vector; retrieves historical interaction records and product information related to the current interaction scenario from the preset knowledge base to generate a knowledge feature vector; and uses the attention fusion mechanism to combine the initial feature vector with the knowledge feature vector to generate a combined embedding vector.

[0010] A multi-level context window is constructed, and in each context window, multiple key indicators of the combined embedding vector and the historical interaction information in the window are calculated respectively; a decreasing attenuation function is designed based on the cross-correlation between the key indicators, and weight coefficients are assigned to context information of different time scales to generate a dynamic weight matrix.

[0011] The combined embedding vector and the dynamic weight matrix are input into a large language model to extract user intent features; the user intent features are multi-headed analyzed based on an attention mechanism to obtain an explicit intent vector, an implicit intent vector and a sentiment tendency vector respectively; the explicit intent vector, the implicit intent vector and the sentiment tendency vector are fused to generate a user feature portrait.

[0012] As a preferred solution of the LLM-based customer intention identification and response system described in the present invention, the key indicators include semantic similarity, sentiment volatility and topic coherence; the semantic similarity is obtained by calculating the cosine distance between the combined embedding vector and the historical interaction information; the sentiment volatility is obtained by analyzing the variance of sentiment polarity in N consecutive interactions; and the topic coherence is obtained by calculating the similarity of topic vectors in adjacent rounds of interactions.

[0013] Designing a decreasing attenuation function based on the cross-correlation between the key indicators includes: designing a dynamic adjustment rule based on the cross-correlation of the semantic similarity, emotional volatility and topic coherence, specifically, when the semantic similarity and topic coherence both show an upward trend, and the emotional volatility shows a downward trend, it is determined to be a high concentration state, and the attention enhancement factor is calculated. Based on the attention enhancement factor, the shape parameter of the time attenuation curve is updated, the sliding window method is used to recalculate the distribution ratio of the weight of each dimension, and the attention state matrix is ​​updated.

[0014] When the sentiment fluctuation rate shows an upward trend, and the semantic similarity and topic coherence both show a downward trend, it is determined to be in a mood fluctuation state, and a mood smoothing function is constructed. Based on the mood smoothing function, the shape parameters of the time decay curve are adjusted, the exponential moving average of historical information is introduced as a stabilizing term, and a temporary weight buffer is established.

[0015] When the topic coherence shows a downward trend but the semantic similarity remains at a high level and the sentiment volatility fluctuates, it is determined to be a topic shift state, the topic migration coefficient is calculated, the shape parameters of the time decay curve are corrected based on the topic migration coefficient, the weight update speed is adjusted using an adaptive learning rate, and a topic coherence compensation mechanism is established.

[0016] A decreasing attenuation function is generated based on the dynamic adjustment rule; the decreasing attenuation function includes a time weight coefficient, a semantic similarity weight, an emotional weight coefficient and a topic coherence weight, wherein each weight coefficient satisfies a normalization constraint and is non-negative; the decreasing attenuation function also includes a time attenuation rate, a baseline attenuation rate and a modulation function corresponding to each state, wherein the modulation function is used to adjust the attenuation curve under different states.

[0017] As a preferred solution of the LLM-based customer intention recognition and response system described in the present invention, wherein: the reinforcement learning dialogue strategy generation unit constructs a multi-objective state value function based on the user feature portrait and generates a dialogue strategy decision sequence, specifically mapping the explicit intention vector, implicit intention vector and emotional tendency vector to state values ​​of three dimensions: dialogue goal completion, user satisfaction and emotional stability; based on the weight distribution ratio in the dynamic weight matrix, the state values ​​of the three dimensions are weighted combined to construct a multi-objective state value function; the Monte Carlo tree search algorithm is used to plan strategies for the next N rounds of dialogues, and the action sequence that maximizes the multi-objective state value function is selected as the dialogue strategy decision sequence.

[0018] Based on the dialogue strategy decision sequence, multiple rounds of dialogue response content are generated, and a dynamic feedback calibration mechanism is established, specifically, the dialogue strategy decision sequence is matched with the product information in the preset knowledge base to generate a candidate response set; the time decay rate in the dynamic weight matrix is ​​used to score the timeliness of the historical response templates in the candidate response set; according to the output value of the emotion smoothing function, the intensity coefficient of the response tone is adjusted; based on the topic migration coefficient, the rhythm and depth of topic switching are controlled; the response content with the highest score is used as the output result of the current round, and the deviation between the response result and the actual interaction effect is fed back to the state value calculation process to optimize the multi-objective state value function.

[0019] As a preferred solution of the LLM-based customer intention recognition and response system described in the present invention, wherein: the multimodal output unit receives the response content, and based on a preset multimodal mapping rule, converts the text content into multimodal output information in three modes of voice, expression and body movement, and according to the real-time scene parameters, adaptively adjusts the multimodal output information, synchronously integrates the adjusted multimodal output information, and outputs the final interactive response; the real-time scene parameters include pitch, speech speed and volume; the multimodal output information includes voice data and visualization information.

[0020] To further solve the above technical problems, the present invention provides the following technical solutions: a customer intention recognition and response method based on LLM, comprising: collecting multi-source interaction data and fusing them to generate a multi-dimensional input representation; performing context understanding based on the multi-dimensional input representation and a preset knowledge base to generate a user feature portrait; dynamically generating a dialogue strategy based on the user feature portrait, and generating response content based on the dialogue strategy; converting the response content into multimodal output information, and adaptively adjusting the multimodal output information according to real-time scene parameters.

[0021] A computer device includes a memory and a processor, wherein the memory stores a computer program, and wherein the processor implements the steps of the LLM-based customer intent identification and response system when executing the computer program.

[0022] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the LLM-based customer intent identification and response system are implemented as described above.

[0023] Beneficial effects of the present invention: The present invention realizes all-round, dynamic recognition and precise response to customer intentions by constructing a system architecture of a multimodal input processing unit, a dynamic context-enhanced LLM unit, a reinforcement learning dialogue strategy generation unit and a multimodal output unit: through multi-channel data acquisition and multi-level feature fusion, the user's acoustic, linguistic, rhythmic and environmental characteristics are fully captured; the user's attention state and emotional changes are accurately tracked by using a dynamic weight matrix and a decreasing attenuation function; based on a multi-target state value function and a dynamic feedback calibration mechanism, the optimal dialogue strategy and response content are generated; through adaptive adjustment of multimodal output, the naturalness of the system response and the interactive experience are ensured, thereby significantly improving the performance of the intelligent dialogue system in actual application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0025] Figure 1 This is a schematic diagram of the structure of the LLM-based customer intention recognition and response system proposed in the present invention;

[0026] Figure 2 A computer device diagram of the LLM-based customer intent recognition and response system proposed in the present invention. DETAILED DESCRIPTION

[0027] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0028] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0029] Example 1, reference Figure 1 , which is an embodiment of the present invention, provides a customer intention recognition and response system based on LLM.

[0030] Existing intelligent dialogue systems use a single modal processing method and fixed dialogue templates, which makes it difficult to accurately understand user intentions and generate flexible response content; and traditional systems use a fixed-window context memory mechanism that cannot dynamically track changes in users' cognitive states and attention distribution, resulting in deviations between dialogue strategies and actual user needs; in addition, existing technologies lack an effective fusion and collaborative optimization mechanism for multimodal features, and rarely consider real-time scene parameters in multimodal output, affecting the naturalness of system responses and the interactive experience.

[0031] The present application provides an effective solution to the above-mentioned problems. Next, multiple embodiments will be combined to explain in detail how to implement the LLM-based customer intent recognition and response system.

[0032] Figure 1 The structural schematic diagram of the LLM-based customer intention recognition and response system provided by the present invention includes a multimodal input processing unit, a dynamic context enhanced LLM unit, a reinforcement learning dialogue strategy generation unit, and a multimodal output unit.

[0033] The multimodal input processing unit collects and fuses multi-source interaction data of the intelligent voice outbound call system to form a multi-dimensional input representation.

[0034] Specifically, the multimodal input processing unit obtains multi-source interactive data in the intelligent voice outbound call system in real time through the multi-channel data acquisition module, including: obtaining the acoustic characteristics of the customer's voice through the audio acquisition channel, including acoustic parameters such as fundamental frequency, resonance peak, and spectral envelope; transcribing the customer's voice into text information in real time through the speech recognition engine, and extracting linguistic features such as keywords, parts of speech, and syntactic structure in the text; extracting prosodic features in the speech through the emotion recognition unit, including prosodic parameters such as pitch, sound intensity, speaking speed, and pauses, which are used to characterize the speaker's emotional state; obtaining background noise characteristics during the call through the environmental acoustic analysis unit, including acoustic environmental parameters such as signal-to-noise ratio, noise type, and reverberation degree.

[0035] In order to effectively fuse the above-mentioned multi-source interaction data, the multimodal input processing unit adopts a multi-level feature fusion strategy: first, the various types of original features obtained (acoustic features, linguistic features, prosodic features, and background noise features) are normalized and preprocessed to make features of different dimensions comparable; second, the attention mechanism is used to weight the importance of different modal features to highlight the features that are more critical to the current interaction state; then, the weighted features are mapped to a unified feature space through a multimodal feature encoder to generate a feature representation that is aligned between modalities; finally, a cross-modal fusion network is used to organically integrate the features of each modality through the complementarity and synergy between modalities to form a multi-dimensional input representation that can comprehensively characterize the current interaction state.

[0036] The specific fusion process can be expressed as:

[0037] 1. Feature normalization. Normalize the original features of each modality to a unified range through methods such as min-max or z-score;

[0038] 2. Attention weighting. Calculate the importance weight of each feature in the current interaction scenario to form a dynamic feature weight matrix;

[0039] 3. Feature encoding. Use a modality-specific encoder to convert raw features into a latent representation of uniform dimension;

[0040] 4. Cross-modal fusion: Through the designed fusion network structure, the correlation and complementarity between modalities are captured to generate the final multi-dimensional representation.

[0041] Preferably, the multimodal input processing unit takes into account the particularity of the intelligent outbound call scenario, comprehensively collects multi-dimensional features such as acoustics, linguistics, emotions, and environment; adopts a dynamic attention mechanism, which can adaptively adjust the importance weight of features according to the interaction scenario; and designs a special cross-modal fusion network to effectively handle the heterogeneity and correlation between modalities. Through this multi-level feature extraction and fusion strategy, it can provide rich and accurate input information for subsequent context understanding and strategy generation, and improve the system's intelligent interaction capabilities.

[0042] The dynamic context-enhanced LLM unit performs context understanding based on multi-dimensional input representation and preset knowledge base to generate user feature profiles.

[0043] Firstly, the dynamic context-enhanced LLM unit obtains the multi-dimensional input representation and inputs it into the pre-trained feature conversion network to obtain the initial feature vector; then the historical interaction records and product information related to the current interaction scenario are retrieved from the preset knowledge base to generate the knowledge feature vector; and then the initial feature vector is combined with the knowledge feature vector using the attention fusion mechanism to generate a combined embedding vector.

[0044] It should be noted that the preset knowledge base includes historical interaction data, product knowledge graphs and the latest market trend information.

[0045] Secondly, a multi-level context window is constructed, and in each context window, multiple key indicators of the combined embedding vector and the historical interaction information in the window are calculated respectively. A decreasing attenuation function is designed based on the cross-correlation between the key indicators, and weight coefficients are assigned to the context information at different time scales to generate a dynamic weight matrix.

[0046] Specifically, the key indicators include semantic similarity, sentiment volatility, and topic coherence. Semantic similarity is obtained by calculating the cosine distance between the combined embedding vector and the historical interaction information; sentiment volatility is obtained by analyzing the variance of sentiment polarity in N consecutive interactions; and topic coherence is obtained by calculating the similarity of topic vectors in adjacent rounds of interactions.

[0047] Furthermore, the design of a decreasing decay function based on the cross-correlation between key indicators includes:

[0048] Firstly, a dynamic adjustment rule is designed based on the cross-correlation of semantic similarity, sentiment fluctuation and topic coherence. Specifically, when both semantic similarity and topic coherence show an upward trend, and sentiment fluctuation shows a downward trend, it is judged as a high-attention state, and the attention enhancement factor is calculated. Based on the attention enhancement factor, the shape parameters of the time decay curve are updated, the sliding window method is used to recalculate the distribution ratio of the weights of each dimension, and the attention state matrix is ​​updated.

[0049] When the sentiment fluctuation rate shows an upward trend, and the semantic similarity and topic coherence both show a downward trend, it is determined to be a sentiment fluctuation state, and a sentiment smoothing function is constructed. Based on the sentiment smoothing function, the shape parameters of the time decay curve are adjusted, the exponential moving average of historical information is introduced as a stabilizing term, and a temporary weight buffer is established;

[0050] When the topic coherence shows a downward trend but the semantic similarity remains at a high level and the sentiment volatility fluctuates, it is determined to be a topic shift state, and the topic migration coefficient is calculated. The shape parameters of the time decay curve are corrected based on the topic migration coefficient, the adaptive learning rate is applied to adjust the weight update speed, and a topic coherence compensation mechanism is established.

[0051] It should be noted that the attention enhancement factor is obtained by calculating the weighted combination of semantic similarity, the rising slope of topic coherence and the falling slope of sentiment volatility, and is used to quantify the user's attention concentration in the current interaction. The topic migration coefficient is obtained by calculating the ratio between the rate of decline of topic coherence and the stability of semantic similarity, and is used to characterize the user's tendency to switch topics while maintaining semantic understanding. The sentiment smoothing function is calculated by performing an exponentially weighted moving average on sentiment volatility, and is used to reduce the sudden impact of sentiment fluctuations on weight distribution.

[0052] Secondly, a decreasing decay function is generated based on dynamic adjustment rules. The decreasing decay function includes time weight coefficient, semantic similarity weight, sentiment weight coefficient and topic coherence weight, where each weight coefficient satisfies the normalization constraint and is non-negative; the decreasing decay function also includes time decay rate, baseline decay rate and modulation function corresponding to each state, where the modulation function is used to adjust the decay curve in different states. According to the decreasing decay function, weight coefficients are assigned to context information of different time scales to generate a dynamic weight matrix.

[0053] Preferably, the present invention can accurately capture the dynamic changes of the user's attention state in actual application scenarios by constructing a multi-level context window and designing a decreasing decay function based on the cross-correlation of the three key indicators of semantic similarity, emotional volatility and topic coherence. In particular, by combining the three typical states of high concentration, emotional fluctuations and topic deviation with the corresponding dynamic adjustment mechanism, the technical problem of the difficulty in accurately identifying and timely responding to the user's cognitive state transition in the prior art is effectively solved. This solution not only significantly improves the adaptability to complex scenarios such as distracted attention and emotional fluctuations in the long-term consultation process of users, but also ensures the coherence and stability of the entire process of the dialogue system in actual application through the dynamic weight allocation mechanism of the decreasing decay function.

[0054] Finally, the combined embedding vector and the dynamic weight matrix are input into the large language model to extract the user intent features. The user intent features are multi-headed analyzed based on the attention mechanism to obtain the explicit intent vector, implicit intent vector and sentiment tendency vector respectively. The explicit intent vector, implicit intent vector and sentiment tendency vector are fused to generate a user feature portrait.

[0055] The reinforcement learning dialogue strategy generation unit dynamically generates dialogue strategies according to the user feature profile, and generates response content based on the dialogue strategies.

[0056] First, the reinforcement learning dialogue strategy generation unit constructs a multi-objective state-value function based on the user feature profile and generates a dialogue strategy decision sequence.

[0057] Specifically, the explicit intention vector, implicit intention vector and emotional tendency vector are mapped to the state values ​​of the three dimensions of dialogue goal completion, user satisfaction and emotional stability respectively; based on the weight distribution ratio in the dynamic weight matrix, the state values ​​of the three dimensions are weighted combined to construct a multi-objective state value function; the Monte Carlo tree search algorithm is used to plan the strategy for the next N rounds of dialogue, and the action sequence that maximizes the multi-objective state value function is selected as the dialogue strategy decision sequence.

[0058] Secondly, multi-round dialogue response content is generated based on the dialogue strategy decision sequence, and a dynamic feedback calibration mechanism is established.

[0059] Specifically, the dialogue strategy decision sequence is matched with the product information in the preset knowledge base to generate a set of candidate responses; the time decay rate in the dynamic weight matrix is ​​used to score the timeliness of the historical response templates in the candidate response set; the intensity coefficient of the response tone is adjusted according to the output value of the emotion smoothing function; the rhythm and depth of topic switching are controlled based on the topic migration coefficient; the response content with the highest score is used as the output result of the current round, and the deviation between the response result and the actual interaction effect is fed back to the state value calculation process to optimize the multi-objective state value function.

[0060] Preferably, the reinforcement learning dialogue strategy generation unit organically integrates the dynamic weight matrix and user feature profile output by the dynamic context-enhanced LLM unit into the reinforcement learning framework, realizing dynamic tracking and response to the user's cognitive state during the dialogue strategy generation process. The dialogue goal, user satisfaction, and emotional stability are uniformly modeled through a multi-objective state value function, and the dynamic weight matrix is ​​used to achieve an adaptive trade-off between these three goals. In addition, in the intelligent outbound call scenario, the system can automatically adjust the dialogue strategy according to the user's attention state, emotional changes, and topic preferences, making the product recommendation process more natural and smooth.

[0061] Table 1 Performance comparison and evaluation of intelligent outbound call systems based on dynamic context enhancement

[0062] Evaluation Metrics Traditional fixed strategy The present invention Improvement Average conversation duration (seconds) 127 159 25.2％ Transaction conversion rate (%) 4.2 4.83 15.0％ User interruption rate (%) 28.5 19.9 －30.2％ User satisfaction rating (1-5 points) 3.6 4.1 13.9％ Topic completeness (%) 62.3 78.6 26.2％ Negative emotion recognition accuracy (%) 71.2 86.7 21.8％ Dynamic response delay (ms) 850 720 －15.3％ Cost per call (yuan) 2.8 2.95 5.4％

[0063] As shown in Table 1, compared with the traditional fixed strategy, this solution increases the average user conversation duration by 25.2%, increases the transaction conversion rate by 15.0%, and reduces the user interruption rate by 30.2%. This improvement is mainly due to the system's ability to capture and respond to user status changes in a timely manner, reflecting the synergistic effect of dynamic context enhancement and reinforcement learning strategy generation.

[0064] The multimodal output unit converts the response content into multimodal output information and adaptively adjusts the multimodal output information according to real-time scene parameters.

[0065] First, the multimodal output unit receives the response content and converts the text content into output information in three modes: speech, expression, and body movement based on preset multimodal mapping rules.

[0066] Specifically, a pre-trained Chinese TTS model is used to convert text into speech signals; a matching expression sequence is selected from a preset expression action library based on the semantic and emotional labels of the response content; and corresponding body movement control instructions are generated based on the tone intensity coefficient of the response content.

[0067] Then, the multimodal output information is adaptively adjusted according to the real-time scene parameters.

[0068] Specifically, the voice output volume is dynamically adjusted based on the ambient noise level; the frequency of facial expression changes is adjusted according to the user's attention state; and the appropriate combination of body movements is selected according to the type of call scenario.

[0069] Finally, the adjusted multimodal output information is synchronously integrated to output the final interactive response.

[0070] Specifically, a timing alignment algorithm is used to ensure the coordination of the output timing of speech, expression and action; the preset modal fusion rules are used to eliminate the conflicts between different modalities; and the integrated output information is passed to the corresponding execution module.

[0071] In summary, the present invention realizes all-round and dynamic recognition and precise response to customer intentions by constructing a system architecture of a multimodal input processing unit, a dynamic context-enhanced LLM unit, a reinforcement learning dialogue strategy generation unit and a multimodal output unit: through multi-channel data acquisition and multi-level feature fusion, the user's acoustic, linguistic, prosodic and environmental characteristics are fully captured; the user's attention state and emotional changes are accurately tracked by using a dynamic weight matrix and a decreasing attenuation function; based on a multi-objective state value function and a dynamic feedback calibration mechanism, the optimal dialogue strategy and response content are generated; through adaptive adjustment of multimodal output, the naturalness of the system response and the interactive experience are ensured, thereby significantly improving the performance of the intelligent dialogue system in actual application scenarios.

[0072] Embodiment 2 is an embodiment of the present invention, which provides a method for identifying and responding to customer intentions based on LLM, including:

[0073] S1: Collect multi-source interaction data and fuse them to generate multi-dimensional input representation;

[0074] S2: Perform context understanding based on the multi-dimensional input representation and the preset knowledge base to generate a user feature profile;

[0075] S3: dynamically generating a dialogue strategy according to the user feature profile, and generating response content based on the dialogue strategy;

[0076] S4: Convert the response content into multimodal output information, and adaptively adjust the multimodal output information according to real-time scene parameters.

[0077] Example 3, reference Figure 2 , is an embodiment of the present invention, which is different from the previous embodiment in that: if the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0078] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0079] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0080] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0081] Example 4 is an embodiment of the present invention, which provides a customer intention recognition and response system based on LLM. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0082] In order to verify the performance of the present invention in actual application scenarios, the intelligent outbound call system of a customer service center of an insurance company was selected as the test platform, and 1,000 customers were randomly selected as test subjects, and they were evenly divided into 5 groups, and different system configurations were used for control tests. The test environment is a standard call center soundproof room, and the ambient noise is controlled below 45dB. The system hardware configuration uses Intel Xeon E5-2680v4 processor, 384GB memory, and is equipped with NVIDIA A100 GPU for deep learning model acceleration.

[0083] In the multimodal input processing unit, the customer's voice signal is collected through a high-fidelity microphone array with a sampling rate of 16kHz, and the Wav2Vec 2.0 model is used to extract acoustic features, including 13-dimensional acoustic parameters such as pitch, energy, and spectral centroid. Speech recognition uses a real-time transcription engine based on the Transformer architecture, and the word error rate (WER) is controlled below 4.2%. The emotion recognition module extracts rhythmic features based on the ResNet-18 network, including speech rate, pauses, pitch changes, and other features. Environmental acoustic analysis uses an adaptive noise suppression algorithm to achieve an accuracy rate of more than 95% in background noise feature extraction.

[0084] The feature fusion strategy adopts a three-layer attention network. The first layer performs intra-modal feature normalization and maps features of different scales to the interval [-1, 1]. The second layer calculates feature weights through the self-attention mechanism, with acoustic feature weight 0.3, language feature weight 0.4, emotional feature weight 0.2, and environmental feature weight 0.1. The third layer uses a cross-attention mechanism to achieve inter-modal feature alignment and generate a 768-dimensional unified feature vector.

[0085] The dynamic context-enhanced LLM unit uses the improved GPT-3 model as the basic architecture, and the context window size is dynamically adjusted to [128, 512]. In the decreasing decay function, the baseline decay rate is set to 0.85, and the time decay curve uses a modified exponential function. The attention state judgment threshold is set to: semantic similarity> 0.75, emotional fluctuation rate< 0.15, and topic coherence> 0.8 when it is judged as a high attention state.

[0086] The improved PPO algorithm is used for reinforcement learning strategy generation, and the reward function weight distribution is: dialogue goal completion 0.4, user satisfaction 0.4, and emotional stability 0.2. The Monte Carlo tree search depth is set to 4 rounds, with 100 samples per round. The response generation adopts the kernel sampling method with a temperature coefficient of 0.7.

[0087] Table 2 Comparison of performance indicators under different system configurations

[0088]

[0089] Table 3 System adaptability test results in different scenarios

[0090]

[0091] The following conclusions can be drawn from the analysis of experimental data: First, in terms of core performance indicators, the complete system of the present invention has achieved significant improvements compared with traditional single-modal systems and basic LLM systems. The accuracy of intention recognition has increased by 21.8 percentage points to 94.3%, which is due to the effectiveness of the multimodal feature fusion strategy; the response relevance has increased by 24.5 percentage points to 92.8%, verifying the important role of the dynamic context enhancement mechanism; user satisfaction and conversation completion rate have increased by 24.7 and 23 percentage points respectively, fully reflecting the advantages of reinforcement learning strategy generation.

[0092] In addition, the present invention has made a breakthrough in system response delay, reducing the average response delay from 850ms to 450ms, while the system resource occupancy rate only increased by 23 percentage points, indicating that the present invention has significant advantages in optimizing algorithm efficiency. In the adaptability test of different scenarios, even in complex background environments, the system can still maintain a high feature fusion accuracy (86.9%) and strategy generation efficiency (85.8%), proving that the present invention has good environmental adaptability and robustness.

[0093] By cross-analyzing the data in Table 2 and Table 3, it can be found that the performance degradation of the present invention is significantly smaller than that of the control group when dealing with complex scenes such as high noise and emotional fluctuations. For example, when transitioning from a quiet indoor environment to a complex background environment, the feature fusion accuracy only drops by 8.3 percentage points, while the traditional system usually experiences a performance drop of 15-20 percentage points under similar conditions. This fully demonstrates that the dynamic context enhancement mechanism and adaptive adjustment strategy proposed in the present invention have significant technical advantages. The system stability has always remained above 95%, indicating that the present invention has high reliability and practical value in engineering implementation.

[0094] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. LLM-based customer intention recognition and response system, characterized by: include: A multimodal input processing unit, which is used to collect and fuse multi-source interaction data of the intelligent voice outbound call system to form a multi-dimensional input representation; A dynamic context-enhanced LLM unit, which performs context understanding based on the multi-dimensional input representation and a preset knowledge base to generate a user feature profile; A reinforcement learning dialogue strategy generation unit, used to dynamically generate a dialogue strategy according to the user feature profile, and generate response content based on the dialogue strategy; The multimodal output unit is used to convert the response content into multimodal output information and adaptively adjust the multimodal output information according to real-time scene parameters.

2. The LLM-based customer intention recognition and response system according to claim 1, characterized in that: The multi-modal input processing unit acquires multi-source interaction data in the intelligent voice outbound call system in real time through a multi-channel data acquisition module, specifically: Acquire the acoustic characteristics of the customer's voice through the audio collection channel; The voice recognition engine transcribes the customer's voice into text information in real time and extracts the linguistic features in the text; Extract the prosodic features in speech through the emotion recognition module; The background noise characteristics during the call are obtained through the environmental acoustic analysis unit.

3. The LLM-based customer intention recognition and response system as claimed in claim 2, characterized in that: The multimodal input processing unit adopts a multi-level feature fusion strategy to fuse the acquired multi-source interaction data, specifically: Normalizing the semantic features, linguistic features, prosodic features, and background noise features; Weight the importance of different modal features; The weighted features are mapped to a unified feature space through a multimodal feature encoder to generate inter-modality aligned feature representations. A cross-modal fusion network is used to organically integrate the features of each modality through the complementarity and synergy between modalities to form a multi-dimensional input representation.

4. The LLM-based customer intention recognition and response system as claimed in claim 3, characterized in that: The process of generating the user feature portrait includes the following steps: The dynamic context enhanced LLM unit obtains the multi-dimensional input representation and inputs it into a pre-trained feature conversion network to obtain an initial feature vector; retrieves historical interaction records and product information related to the current interaction scenario from the preset knowledge base to generate a knowledge feature vector; and uses an attention fusion mechanism to combine the initial feature vector with the knowledge feature vector to generate a combined embedding vector; Construct a multi-level context window, and calculate multiple key indicators of the combined embedding vector and the historical interaction information in the window in each context window; design a decreasing decay function based on the cross-correlation between the key indicators, assign weight coefficients to context information of different time scales, and generate a dynamic weight matrix; Inputting the combined embedding vector and the dynamic weight matrix into a large language model to extract user intent features; Based on the attention mechanism, the user intention features are multi-headedly analyzed to obtain an explicit intention vector, an implicit intention vector and a sentiment tendency vector respectively; the explicit intention vector, the implicit intention vector and the sentiment tendency vector are fused to generate a user feature portrait.

5. The LLM-based customer intention recognition and response system as claimed in claim 4, characterized in that: The key indicators include semantic similarity, sentiment volatility and topic coherence; The semantic similarity is obtained by calculating the cosine distance between the combined embedding vector and the historical interaction information; The sentiment fluctuation rate is obtained by analyzing the variance of sentiment polarity in N consecutive interactions; The topic coherence is obtained by calculating the similarity of topic vectors of adjacent rounds of interactions; Designing a decreasing decay function based on the cross-correlation between the key indicators includes: Based on the cross-correlation of the semantic similarity, the emotional volatility and the topic coherence, a dynamic adjustment rule is designed. Specifically, when the semantic similarity and the topic coherence both show an upward trend, and the emotional volatility shows a downward trend, it is determined to be a high-focus state, and the attention enhancement factor is calculated. Based on the attention enhancement factor, the shape parameter of the time decay curve is updated, the sliding window method is used to recalculate the distribution ratio of the weight of each dimension, and the attention state matrix is ​​updated; When the sentiment fluctuation rate shows an upward trend, and the semantic similarity and topic coherence both show a downward trend, it is determined to be a sentiment fluctuation state, and a sentiment smoothing function is constructed. Based on the sentiment smoothing function, the shape parameters of the time decay curve are adjusted, the exponential moving average of historical information is introduced as a stabilizing term, and a temporary weight buffer is established; When the topic coherence shows a downward trend but the semantic similarity remains at a high level, and the sentiment volatility fluctuates, it is determined to be a topic shift state, the topic migration coefficient is calculated, the shape parameter of the time decay curve is corrected based on the topic migration coefficient, the weight update speed is adjusted by applying an adaptive learning rate, and a topic coherence compensation mechanism is established; A decreasing attenuation function is generated based on the dynamic adjustment rule; the decreasing attenuation function includes a time weight coefficient, a semantic similarity weight, an emotional weight coefficient and a topic coherence weight, wherein each weight coefficient satisfies a normalization constraint and is non-negative; the decreasing attenuation function also includes a time attenuation rate, a baseline attenuation rate and a modulation function corresponding to each state, wherein the modulation function is used to adjust the attenuation curve under different states.

6. The LLM-based customer intention recognition and response system according to claim 5, characterized in that: The reinforcement learning dialogue strategy generation unit constructs a multi-objective state value function based on the user feature portrait and generates a dialogue strategy decision sequence, specifically mapping the explicit intention vector, implicit intention vector and emotional tendency vector to state values ​​of three dimensions: dialogue goal completion, user satisfaction and emotional stability; based on the weight distribution ratio in the dynamic weight matrix, weighted combination of the state values ​​of the three dimensions is performed to construct a multi-objective state value function; Monte Carlo tree search algorithm is used to plan strategies for the next N rounds of dialogue, and the action sequence that maximizes the multi-objective state value function is selected as the dialogue strategy decision sequence; Generate multiple rounds of dialogue response content based on the dialogue strategy decision sequence, and establish a dynamic feedback calibration mechanism, specifically matching the dialogue strategy decision sequence with product information in a preset knowledge base to generate a candidate response set; using the time decay rate in the dynamic weight matrix to perform timeliness scoring on the historical response templates in the candidate response set; According to the output value of the emotion smoothing function, adjusting the intensity coefficient of the response tone; Based on the topic migration coefficient, the rhythm and depth of topic switching are controlled; the response content with the highest score is used as the output result of the current round, and the deviation between the response result and the actual interaction effect is fed back to the state value calculation process to optimize the multi-objective state value function.

7. The LLM-based customer intention recognition and response system according to claim 6, characterized in that: The multimodal output unit receives the response content, and based on a preset multimodal mapping rule, converts the text content into multimodal output information in three modes: voice, expression, and body movement; according to the real-time scene parameters, adaptively adjusts the multimodal output information, synchronously integrates the adjusted multimodal output information, and outputs a final interactive response; The real-time scene parameters include pitch, speech speed and volume; The multimodal output information includes voice data and visual information.

8. A method for customer intention recognition and response based on LLM, based on the customer intention recognition and response system based on LLM according to any one of claims 1 to 7, characterized in that: include, Collect multi-source interaction data and fuse them to generate multi-dimensional input representation; Based on the multi-dimensional input representation and the preset knowledge base, context understanding is performed to generate a user feature portrait; Dynamically generate a dialogue strategy according to the user feature profile, and generate response content based on the dialogue strategy; The response content is converted into multimodal output information, and the multimodal output information is adaptively adjusted according to real-time scene parameters.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the LLM-based customer intent identification and response method described in claim 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the LLM-based customer intent identification and response method described in claim 8 are implemented.

Images