Adaptive scene intelligent interaction system based on AI

Abstract

The invention, which relates to the technical field of intelligent interaction, discloses an AI-based adaptive scene intelligent interaction system comprising a multi-modal data acquisition module, a modal preprocessing module, a multi-modal embedded coding module, an intention fusion and representation module, a service scene matching module and a service execution and reinforcement learning module. The method comprises the following steps: acquiring multi-modal original data in a user interaction process, including voice signals, text input and user behavior tracks, and synchronously recording an acquisition timestamp; according to the method, through a multi-modal unified embedding and dynamic weighting mechanism, the problem of characteristic dimension imbalance is effectively solved, and the user intention recognition accuracy is improved; meanwhile, reinforcement learning and a multi-factor scoring model are combined, personalized scene matching and dynamic response are achieved, the adaptive capacity and service accuracy of the system in a complex environment are improved, and therefore the stability and user experience of the intelligent interaction system are remarkably optimized.

Description

Technical Field

[0001] The present invention relates to the field of intelligent interaction technology, and in particular to an AI-based adaptive scene intelligent interaction system. Background Art

[0002] AI-based adaptive scenario-based intelligent interaction uses artificial intelligence technology to perceive and understand multimodal user input (such as voice, text, and behavioral characteristics) in real time. It then integrates contextual factors such as historical behavior and environmental characteristics to accurately identify user intent and dynamically match the most appropriate service processing scenario. The system analyzes user needs using deep learning models such as improved BERT and LSTM networks and continuously optimizes interaction strategies using reinforcement learning mechanisms, ultimately achieving intelligent routing, automated response, and service processing. The system possesses self-learning and scenario-adaptive capabilities, enabling real-time responses to new service types and unexpected demands, significantly improving service efficiency and user satisfaction.

[0003] The existing technology has the following deficiencies: In the existing AI-based adaptive scene intelligent interaction system, the imbalance of feature dimensions in multimodal information processing is an easily overlooked but destructive challenge. When the system simultaneously collects multi-source data such as voice, text, and user behavior, due to the differences in data volume, noise sensitivity, and feature expression of each modality, the model may be overly guided by a certain modal feature during the intent recognition process, resulting in the neglect of other key information. Especially in scenarios where the user's expression is vague, the environment is noisy, or the user is emotionally excited, this deviation can easily lead to misjudgment and mismatching of business processes, which may ultimately result in the user's demands not being properly responded to, affecting service quality and even triggering a crisis of trust.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide an AI-based adaptive scene intelligent interaction system, which effectively solves the problem of feature dimension imbalance through multimodal unified embedding and dynamic weighting mechanism, and improves the accuracy of user intention recognition; at the same time, it combines reinforcement learning and multi-factor scoring models to achieve personalized scene matching and dynamic response, and improve the system's adaptability and service accuracy in complex environments, thereby significantly optimizing the stability of the intelligent interaction system and user experience, so as to solve the problems in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention provides the following technical solutions: an AI-based adaptive scene intelligent interaction system, comprising a multimodal data acquisition module, a modal preprocessing module, a multimodal embedding coding module, an intent fusion and representation module, a business scenario matching module, and a business execution and reinforcement learning module:

[0007] Multimodal data acquisition module, which collects multimodal raw data during user interaction, including voice signals, text input, and user behavior trajectories, and simultaneously records the acquisition timestamp;

[0008] The modal preprocessing module performs preprocessing operations on the collected multimodal raw data. It performs noise reduction and voiceprint emotion analysis on voice signals, performs word segmentation and syntactic analysis on text input, and extracts path features and conducts behavior frequency modeling on user behavior trajectories.

[0009] The multimodal embedding coding module performs independent embedding coding on the preprocessed multimodal feature data based on an improved dual-channel deep neural network to generate an embedding vector of uniform dimension.

[0010] The intent fusion and representation module uses the multimodal embedding vector generated by the attention mechanism to construct a user intent representation vector and dynamically monitors the contribution ratio of each modality through a visually adjustable feature weight matrix.

[0011] The business scenario matching module inputs the user intent representation vector into the pre-built business scenario vector space, identifies the matching degree through cosine similarity calculation, and locates the optimal scenario node;

[0012] The business execution and reinforcement learning module calls the scenario executor to trigger the business process response based on the identified scenario nodes, and transmits user feedback data back to the feedback reinforcement model in real time, periodically optimizing the intent recognition and scenario matching accuracy through the deep reinforcement learning algorithm.

[0013] Preferably, when collecting voice signals during user interaction, the method further includes performing spectrum analysis on the audio content and extracting key frequency domain features, including fundamental frequency change rate, average frequency band energy density, and short-time energy envelope curve, while generating multi-dimensional audio tags in combination with the user's physical environment parameters;

[0014] In terms of text input collection, real-time conversation context modeling is performed for each user session to construct a text semantic frame including keyword sequences, sentiment tendency scores, intent confidence intervals, etc.

[0015] For user behavior trajectories, their contact sequence, mouse trajectory angular velocity and dwell time are collected, and a time series behavior model is established through a multi-window sliding average algorithm to provide an accurate data foundation for subsequent embedding processing and behavior intention inversion.

[0016] Preferably, when performing multimodal preprocessing, the speech signal preprocessing uses a voiceprint recognition model based on a convolutional neural network combined with a multi-task learning mechanism to perform multi-label classification prediction on the user's emotional state, and the predicted labels include four categories: anxiety, anger, confusion, and calmness;

[0017] The text input component introduces a contextual attention mechanism to capture the characteristics of emotional transitions across sentences and output an emotional transition identifier. Regarding user behavior trajectories, a trajectory clustering algorithm based on a graph neural network is used to cluster and reduce the dimensionality of high-frequency contact areas and output an index of the user's preferred area.

[0018] All preprocessing results will be normalized before entering the embedding code to ensure the uniform distribution of multimodal inputs and improve the stability and recognition accuracy during subsequent model fusion.

[0019] Preferably, during the embedded coding process, each type of modal data is processed separately through a coding channel, wherein the voice channel adopts a bidirectional GRU encoder and a superimposed position coding mechanism, the text channel adopts a multi-layer Transformer structure for context modeling, and the user behavior channel introduces a temporal convolution module to extract short-term behavior change features;

[0020] When generating an embedding vector of uniform dimension, the projection mapping of each modality embedding vector is dynamically adjusted through a three-channel embedding space alignment mechanism to ensure consistent expressiveness in the shared embedding space. A modality confidence label is also included as an important basis for the subsequent fusion stage.

[0021] Preferably, when performing a multimodal vector fusion operation, the attention mechanism adopts a stacked multi-head attention network, where the first layer focuses on the cross-influence weights between modalities, and the second layer focuses on the temporal dependency of important features within a single modality;

[0022] The user intention representation vector output by the fusion module has a dimension of 78;

[0023] To prevent a single mode from dominating the judgment, a modal balancing factor is introduced to control the modal contribution deviation rate within ±12% after each round of fusion, and the modal weighting coefficient is automatically adjusted through a feedback learning mechanism.

[0024] Preferably, before generating the user intent representation vector, modal feature normalization fusion is performed to improve the stability and recognition accuracy of the model fusion stage through a dynamically adjustable feature transformation mechanism. The specific steps are as follows:

[0025] For each modal embedding vector, the standard deviation normalization method is used to map the eigenvectors of different modalities to a standard normal distribution with a mean of zero and a variance of one in the numerical domain. The calculation expression is as follows:

[0026] , where The original modality embedding vectors, It is The mean of the embedding vectors generated by the modal embedding vectors in the training sample set, It is The standard deviation of the modal embedding vector, is the normalized modality embedding vectors;

[0027] Based on the signal-to-noise ratio calculated from historical sampling in the predefined sample set, the expression credibility of each modal embedding vector is quantified. By comparing the energy ratio of useful feature signals and noise interference, the importance weight of the modal embedding vector is calculated. The formula is as follows:

[0028] , where It is The signal-to-noise ratio of the modal embedding vector, It is The signal-to-noise ratio of the modal embedding vector, is the total number of modes, It is Importance weights of the modality embedding vectors;

[0029] All normalized embedding vectors are fused using the modality weighted average mechanism to generate a fused comprehensive intent vector. The calculation expression is as follows:

[0030] , where is the final integrated user intention vector;

[0031] The obtained user intention comprehensive vector Input into the subsequent attention fusion module to extract deeper semantic dependency features.

[0032] Preferably, when performing a business scenario matching operation, the business scenario vector space is composed of a 78-dimensional vector. Each scenario node is generated through prior data training, and its feature vector contains four core features: business type identifier, scenario risk level, user historical satisfaction mean, and context behavior similarity;

[0033] The matching mechanism uses real-time cosine similarity calculation. If the similarity between the current user intention vector and multiple scene nodes is higher than 0.85, the context judgment module is automatically started. The optimal scene node is selected based on the weighted decision of the business frequency priority, user identity label and emotional state score at the current time point.

[0034] Preferably, when calculating the scene matching degree, an advanced scoring mechanism that integrates user state perception and scene structure characteristics is introduced to improve the accuracy of the matching results and the real-time performance of the business response. The specific steps are as follows:

[0035] The user's current intention vector is calculated by cosine similarity with each business scenario node vector predicted and trained to generate an initial semantic similarity score. The formula is as follows:

[0036] , where It is The similarity score between the business scenario and the user's current intention, is the user intention representation vector, It is Business scenario node vectors, They are vectors and The Euclidean norm of ;

[0037] The user's current emotion value and the urgency factor corresponding to the business scenario are introduced. The two are jointly used to construct the scenario response priority. The influence weight on the final score is controlled by adjusting the factor. The calculation expression is as follows:

[0038] , where is the scenario response priority, is the emotion weight coefficient, is the user’s current sentiment value, is the scenario urgency weight coefficient, It is Risk or urgency score of each scenario node;

[0039] Adding a time-sensitive factor to characterize the real-time urgency of the scene response, the final scene synthesis is calculated as follows:

[0040] , where is the final scene comprehensive score, is the time sensitivity weight coefficient, is the natural base, Is the current time for The input variables of the urgency function of each business scenario;

[0041] Select the one with the highest The scene node is taken as the final interaction target, and the scene executor is immediately scheduled to trigger the corresponding business process. At the same time, the scoring factors and user response feedback data are synchronously transmitted back to the learning engine for subsequent strategy optimization.

[0042] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0043] The present invention significantly improves the accuracy of user intent recognition by introducing multimodal unified dimension embedding and modal dynamic weighting mechanism. After collecting voice, text and behavioral data, the system suppresses the deviation caused by the imbalance of feature dimensions through standardization and modal confidence-guided attention fusion strategy. Especially in scenarios where user input information is incomplete or environmental interference is significant, the system can automatically identify the quality of the current modality and adjust its weight distribution in the fusion process, so that the final generated user intent representation is more realistic and comprehensive. This mechanism effectively avoids the misjudgment caused by the excessive dominance of a certain modality and significantly reduces the probability of mismatching scenarios.

[0044] This invention combines a reinforcement learning feedback mechanism with a multi-factor scoring model to construct a self-evolving scenario matching system. Based on the similarity between user intent vectors and business scenario vectors, the system further incorporates factors such as sentiment, response urgency, and access timing for weighted correction, thereby achieving a more personalized and dynamic matching process. This structure ensures that the system can accurately identify the most appropriate business scenario node even when faced with complex scenarios and uncertain user expressions, enhancing the automation and intelligence level of the overall business process, and ultimately improving the timeliness and accuracy of service responses. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction to the drawings required for use in the embodiments will be given below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0046] Figure 1 This is a module diagram of the AI-based adaptive scene intelligent interaction system of the present invention. DETAILED DESCRIPTION

[0047] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0048] The present invention provides Figure 1 The AI-based adaptive scenario intelligent interaction system shown in the figure includes a multimodal data acquisition module, a modal preprocessing module, a multimodal embedding coding module, an intent fusion and representation module, a business scenario matching module, and a business execution and reinforcement learning module:

[0049] Multimodal data acquisition module, which collects multimodal raw data during user interaction, including voice signals, text input, and user behavior trajectories, and simultaneously records the acquisition timestamp;

[0050] When collecting voice signals during user interactions, the system further performs spectral analysis on the audio content and extracts key frequency domain features, including fundamental frequency change rate, average frequency band energy density, and short-time energy envelope curve. It also generates multi-dimensional audio tags based on the user's physical environment parameters (such as background noise decibel level and sound source direction angle).

[0051] In terms of text input collection, real-time conversation context modeling is performed for each user session to construct a text semantic frame including keyword sequences, sentiment tendency scores, intent confidence intervals, etc.

[0052] For user behavior trajectories, their contact sequence, mouse trajectory angular velocity and dwell time are collected, and a time series behavior model is established through a multi-window sliding average algorithm to provide an accurate data foundation for subsequent embedding processing and behavior intention inversion.

[0053] The above steps enhance the integrity and contextual relevance of multimodal data collected during user interactions. High-dimensional, multi-source input ensures that subsequent AI processing modules acquire an accurate and rich data foundation, addressing the recognition biases caused by isolated data and missing features in traditional systems. In practical systems, implementing this step requires three capabilities: high-fidelity data capture, multi-dimensional label extraction, and inter-modal time synchronization.

[0054] First, voice signal processing isn't limited to capturing raw waveforms. Instead, it should be based on short-time Fourier transform (STFT) spectrograms to extract features such as fundamental frequency variation, frequency band energy density, and voiceprint energy concentration. These features can reveal the user's emotions, speech rate, and intonation, helping AI identify emotional states such as anxiety and anger.

[0055] Secondly, at the text input layer, natural language processing tools (such as HanLP and spaCy) are used for word segmentation and part-of-speech tagging. Semantic vectors are also constructed, sentiment tendencies (such as anger, doubt, and negative meanings) are identified, and keywords are extracted. If a user issues multiple sentences simultaneously, a sentiment jump detection mechanism can be introduced to identify potential sudden changes in demand.

[0056] Finally, user behavior trajectories are sampled from dimensions such as touch trajectory, click hotspot distribution, and mouse angular velocity to construct a sliding window behavior model. Each action is timestamped to ensure alignment with the voice and text data. These trajectory patterns can reveal implicit behavioral intentions, such as confusion (frequent clicking) and hesitation (long hovering).

[0057] Through the complete collection and labeling of the above three types of modalities, the system has a strong raw data input foundation, laying a solid foundation for subsequent high-precision intelligent recognition.

[0058] The modal preprocessing module performs preprocessing operations on the collected multimodal raw data. It performs noise reduction and voiceprint emotion analysis on voice signals, performs word segmentation and syntactic analysis on text input, and extracts path features and conducts behavior frequency modeling on user behavior trajectories.

[0059] When performing multimodal preprocessing, the voice signal preprocessing uses a voiceprint recognition model based on a convolutional neural network combined with a multi-task learning mechanism to perform multi-label classification prediction on the user's emotional state. The predicted labels include anxiety, anger, confusion, and calmness.

[0060] The text input component introduces a contextual attention mechanism to capture the characteristics of emotional transitions across sentences and output an emotional transition identifier. Regarding user behavior trajectories, a trajectory clustering algorithm based on a graph neural network is used to cluster and reduce the dimensionality of high-frequency contact areas and output an index of the user's preferred area.

[0061] All preprocessing results will be normalized before entering the embedding code to ensure the uniform distribution of multimodal inputs and improve the stability and recognition accuracy during subsequent model fusion.

[0062] The above steps enhance the intelligence and robustness of the data preprocessing phase, ensuring that multimodal input data possesses rich semantics, readable emotions, and clear behavioral structure before entering the deep learning model. Traditional AI systems often underemphasize preprocessing, limiting subsequent recognition performance. This approach effectively addresses this shortcoming by integrating emotion recognition, multi-task learning, and graph-structured behavioral modeling.

[0063] First, the speech component uses a CNN model to extract parameters such as Mel-Frequency Cepstral Coefficients (MFCCs), which are then fed into a multi-task learning framework for emotion classification. Emotions are classified into four categories: anxiety, anger, confusion, and calmness. Training using a cross-entropy loss function yields multi-label results, which assist subsequent modules in understanding the user's current state of mind.

[0064] Secondly, the BERT+CRF model is used for text input preprocessing to perform entity recognition and sentiment analysis. The contextual attention mechanism tracks inter-sentence semantic transitions, detecting complex intents such as "I want to cancel but am concerned about the cost," and providing sentiment and conflict annotation for semantic fusion.

[0065] Behavioral trajectory data is clustered by constructing a user behavior graph (such as click nodes as graph nodes and paths as edges) using graph neural networks (GNNs) to identify high-frequency click areas, operation backtracking behaviors, and interface adaptation difficulties, forming a structured behavioral feature vector.

[0066] The above processing results are then normalized (e.g., using the Z-Score or Min-Max method) to ensure that the different modalities are scaled consistently before entering the next step of the embedding modeling process. This step effectively addresses the uneven representation issues caused by modal differences.

[0067] The multimodal embedding coding module independently embeds and codes the pre-processed multimodal feature data based on an improved dual-channel deep neural network, generating an embedding vector of uniform dimension to avoid the dominant effect of one modality in the feature space.

[0068] During the embedding encoding process, each type of modal data is processed separately through encoding channels. The voice channel uses a bidirectional GRU encoder with a superimposed positional encoding mechanism, the text channel uses a multi-layer Transformer structure for context modeling, and the user behavior channel introduces a temporal convolution module to extract short-term behavioral change features.

[0069] When generating an embedding vector of uniform dimension, the projection mapping of each modality embedding vector is dynamically adjusted through a three-channel embedding space alignment mechanism to ensure consistent expressiveness in the shared embedding space. A modality confidence label is also included as an important basis for the subsequent fusion stage.

[0070] The goal of the above steps is to build a specialized encoding path for multimodal data, ensuring consistent representation of voice, text, and behavioral data within a unified embedding space, thereby preventing the risk of any one modality dominating the final model's judgment. This is achieved through three mechanisms: modality separation encoding, cross-modal projection, and confidence annotation.

[0071] First, the speech channel uses a bidirectional GRU (BiGRU) network, which can simultaneously capture forward and backward features of speech data. Positional encoding (such as Sinusoidal Position Encoding) is embedded before encoding to improve word order understanding. This approach enhances the model's ability to recognize emotional changes and shifts in intent in long speech segments.

[0072] Secondly, the text channel uses a multi-layer Transformer architecture (6 or 12 layers) to perform self-attention modeling on the input semantic fragments, capturing language logic such as subject-verb-object structure, negation, and emotional rhetorical questions. Through residual connections and layer normalization, the original syntactic information is preserved, improving the accuracy of semantic encoding.

[0073] The user behavior channel introduces a one-dimensional temporal convolutional network (TCN), which can extract temporal features such as click frequency change trends and reaction speed differences in a short period of time, and generate a highly expressive behavior embedding vector.

[0074] After the above three types of encoding output, they enter a modal vector alignment module, which maps each modality into a shared 78-dimensional semantic space through a learnable linear projection matrix and adds modal confidence (such as speech signal quality score, text completeness, etc.) to provide adjustable trust indicators for the fusion stage.

[0075] The intent fusion and representation module uses the multimodal embedding vector generated by the attention mechanism to construct a user intent representation vector and dynamically monitors the contribution ratio of each modality through a visually adjustable feature weight matrix.

[0076] When performing multimodal vector fusion operations, the attention mechanism uses a stacked multi-head attention network. The first layer focuses on the cross-influence weights between modalities, and the second layer focuses on the temporal dependencies of important features within a single modality.

[0077] The user intent representation vector output by the fusion module has 78 dimensions, each of which represents a specific semantic or behavioral dimension, such as behavioral tension index, language emotion volatility, and semantic complexity score.

[0078] To prevent a single mode from dominating the judgment, a modal balancing factor is introduced to control the modal contribution deviation rate within ±12% after each round of fusion, and the modal weighting coefficient is automatically adjusted through a feedback learning mechanism.

[0079] The purpose of these steps is to fuse the embedding vectors of multiple modalities into a semantically consistent and behaviorally sensitive representation of user intent. By dynamically controlling the modal contribution ratio, this overcomes the problem of a single modality dominating the recognition results in traditional models. This fusion process is designed based on a multi-layer attention mechanism and adaptively controlled through a modality balance factor and policy learning mechanism.

[0080] In terms of implementation, the fusion module consists of a two-layer attention network. The first layer is the inter-modal attention mechanism, which is responsible for learning the correlations between modalities. For example, whether "expressing anxiety in user voice" is similar to "quickly clicking to exit the page" behavior. This mechanism calculates the correlation coefficient between the inter-modal embedding vectors to generate an inter-modal attention weight matrix.

[0081] The second layer is the intra-modal attention mechanism, which is used to deeply explore feature dimensions with high semantic value within a single modality. For example, in the text modality, the attention weight for recognizing keywords such as "cancel," "complain," and "unavailable" is much higher than that for common auxiliary words. This mechanism uses a multi-head self-attention structure to calculate the weight distribution of different positions within a single modality and prioritizes the high-weighted ones for the fusion output.

[0082] To prevent modal dominance, the system introduces a "modal deviation rate" assessment mechanism. If the weight of a particular modality in the fused user intent vector exceeds 60% of the overall weight, a modal balancing process is triggered, adjusting the attention weight of that modality to within ±12% of the average level. This deviation control strategy ensures that all modalities are fairly represented, improving the robustness of the fused vector.

[0083] After the fusion vector is output, it enters the policy optimization module. Deep reinforcement learning methods (such as DQN or PPO) are used to periodically update the fusion strategy based on actual business feedback (such as user satisfaction and the need for manual transfer). This allows the fusion mechanism to continuously evolve based on real-world scenarios, ensuring that the intent vector maintains strong recognition and generalization capabilities for business objectives.

[0084] Before generating the user intent representation vector, modal feature normalization fusion is performed. This dynamically adjustable feature transformation mechanism aims to address structural inconsistencies in distribution density, scale, and numerical domain among multimodal inputs, thereby improving the stability and recognition accuracy of the model fusion phase. The specific steps are as follows:

[0085] For each modal embedding vector, the standard deviation normalization method is used to map the eigenvectors of different modalities to a standard normal distribution with a mean of zero and a variance of one in the numerical domain, thereby eliminating the dominance of the original embedding vector in absolute value. The calculation expression is as follows:

[0086] , where The original A modality embedding vector represents the vector output of a certain modality (such as speech, text, behavior) after processing by the embedding coding module. Its dimension is consistent with the system's intention expression dimension (such as 78 dimensions). It is The mean of the embedding vectors generated by the modal embedding vectors in the training sample set is used to represent the center position of the modal vector distribution and is a historical statistic. It is The standard deviation of the modal embedding vector measures the degree of dispersion of the modal feature distribution; the larger the value, the stronger the volatility of the modality in the feature dimension. is the normalized A modality embedding vector with a standard normal distribution of mean 0 and standard deviation 1 is used to eliminate the interference of inter-modality scale differences on model judgment;

[0087] This process ensures that in the subsequent weighted calculation, the influence of each modality comes from semantic expression rather than numerical size.

[0088] Based on the signal-to-noise ratio calculated from historical sampling in the predefined sample set, the expression credibility of each modal embedding vector is quantified. By comparing the energy ratio of useful feature signals and noise interference, the importance weight of the modal embedding vector is calculated. The formula is as follows:

[0089] , where It is The signal-to-noise ratio of a modal embedding vector is a measure of the ratio of the effectiveness of the modal signal to the degree of interference. For example, for the speech modality, it can be estimated based on the MFCC feature signal and the background noise energy; for the text modality, it can be inferred from the confidence interval deviation of the word vector. It is The signal-to-noise ratio of a modal embedding vector is an indicator that measures the ratio of useful information (signal) to interference information (noise) in a modal data. is the total number of modes, It is The importance weight of each modal embedding vector is used in the subsequent multimodal weighted fusion stage and is the key parameter for balancing the forces of each input source in the decision-making process;

[0090] This weight reflects the relative reliability of each modality in semantic expression under the current environmental conditions, thereby improving the model's adaptability in the case of multi-source heterogeneous input.

[0091] All normalized embedding vectors are fused using the modality weighted average mechanism to generate a fused comprehensive intent vector. The calculation expression is as follows:

[0092] , where It is the final integrated user intent vector, with the same dimension as the unimodal embedding (e.g., 78 dimensions). It integrates the standardized semantic features of all modalities and takes into account the differences in information contribution between modalities.

[0093] This operation ensures that the model can fully balance the asymmetry of information between modalities when comprehensively expressing user intentions.

[0094] The obtained user intention comprehensive vector The data is input into the subsequent attention fusion module to extract deeper semantic dependency features, and finally output an accurate and stable user intent representation vector for use by the scene matching engine.

[0095] The above steps start with the quality of modal input and effectively integrate the data from each modality through standardization and weight adjustment mechanisms, preventing any one modality from dominating and misleading the recognition process. The ultimate goal is to output a comprehensive and well-balanced user intent representation vector that accurately reflects the user's true intent in specific business interaction scenarios.

[0096] During implementation, modal data from various sources, including speech, text, and behavior, is first normalized. The primary purpose of normalization is to eliminate order-of-magnitude differences caused by varying data sources and dimensions, ensuring fair evaluation of all modal inputs. This unified scaling avoids bias toward modalities with larger magnitudes, providing a stable data foundation for subsequent processing.

[0097] On this basis, the system also introduces a modal signal quality assessment mechanism. This mechanism calculates a relative signal-to-noise ratio score for each modality based on indicators such as the historical noise level, stability, and data loss rate of each modality. This score is then used to determine the weight that each modality should occupy in the final fusion vector. For example, during certain interactions, if the system detects that the user's speech is clear but the behavioral data is too sparse, the speech modality will be given a higher influence weight to ensure recognition accuracy.

[0098] After weighted fusion, the system combines all modal inputs into a unified fused vector and uses this fused vector as input to the subsequent attention mechanism. The attention mechanism further enhances the semantic features of the fused vector, ensuring that in the actual semantic space, the fusion result not only preserves the information synergy between the modalities but also has sufficient contextual expression capabilities.

[0099] Through the above process, Claim 7 realizes a multimodal fusion mechanism that takes into account the balance of modal expression, semantic clarity and data credibility, providing strong perception layer support for the intelligent interactive system.

[0100] The business scenario matching module inputs the user intent representation vector into the pre-built business scenario vector space, identifies the matching degree through cosine similarity calculation, and locates the optimal scenario node;

[0101] When performing business scenario matching operations, the business scenario vector space consists of a 78-dimensional vector. Each scenario node is generated through prior data training, and its feature vector contains four core features: business type identifier, scenario risk level, user historical satisfaction mean, and contextual behavior similarity;

[0102] The matching mechanism uses real-time cosine similarity calculation. If the similarity between the current user intention vector and multiple scene nodes is higher than 0.85, the context judgment module is automatically started. The optimal scene node is selected based on the weighted decision of the business frequency priority, user identity label and emotional state score at the current time point.

[0103] The above steps construct a structured business scenario vector space and, incorporating contextual factors, accurately match user intent, ensuring that the final scenario selection meets both user needs and the characteristics of the current business environment. In actual deployments, this step significantly reduces the matching error rate caused by static rules in traditional systems, and is particularly valuable in multi-service convergence platforms.

[0104] The business scenario vector space is constructed based on actual business scenario statistics. Each node represents a business scenario (such as "broadband renewal" or "traffic anomaly diagnosis") and corresponds to a 78-dimensional feature vector. This vector includes a business identifier (such as BID-001), a scenario risk level (low, medium, or high), a historical average satisfaction score (a decimal value between 0 and 1), and a behavioral similarity reference value. Each dimension has a clear physical meaning, ensuring interpretability during the model matching process.

[0105] After generating the user intent representation vector, the system performs a vector cosine similarity calculation on all scene nodes and outputs a ranked list. If the similarity of the top three scene nodes is greater than 0.85, the system will enter the context-based fine matching stage.

[0106] The context matching stage comprehensively evaluates the following three types of weighting factors:

[0107] Frequency of business requests at the current time: For example, complaints are prioritized during peak hours in the evening.

[0108] User identity tag: For example, VIP customers have higher priority;

[0109] User's current emotional state score: For users with extreme emotions, priority is given to scenarios with high processing efficiency.

[0110] These factors are then re-ranked using a weighted scoring mechanism, ultimately selecting the optimal scene node to drive the actuator response. This mechanism takes into account both scene semantic similarity and environmental adaptability, effectively improving matching accuracy and user satisfaction.

[0111] The business execution and reinforcement learning module, based on the identified scenario nodes, calls the scenario executor to trigger the business process response and transmits user feedback data back to the feedback reinforcement model in real time. It uses the deep reinforcement learning algorithm to periodically optimize the accuracy of intent recognition and scenario matching.

[0112] When calculating scene matching, we introduce an advanced scoring mechanism that integrates user state perception and scene structure characteristics. This aims to address the problem that traditional matching models cannot dynamically consider contextual factors (such as emotional fluctuations, scene stress levels, and time sensitivity), thereby improving the accuracy of matching results and the real-time nature of business responses. The specific steps are as follows:

[0113] The user's current intention vector is calculated by cosine similarity with each business scenario node vector predicted and trained to generate an initial semantic similarity score. The formula is as follows:

[0114] , where It is The similarity score between a business scenario and the user's current intention ranges from 0 to 1. It is a user intention representation vector, generated by the system through preprocessing, embedding, and fusing the multimodal data (voice, text, behavior, etc.) input by the user. The 78-dimensional vector; each dimension represents a characteristic expression of the user's current intention (such as semantic strength, behavioral characteristics, or emotional index, etc.), which is the system's precise encoding of the current interaction semantics. It is Business scenario node vectors are obtained by pre-training and represent business process scenarios known to the system, such as "package change" and "network problem complaint". Each scenario vector is also 78-dimensional, and the dimension distribution is the same as Remain consistent, containing information such as business type labels, context preference weights, historical user behavior vectors, etc. They are vectors and The Euclidean norm of is used to calculate the normalized cosine similarity to avoid errors caused by differences in vector length. The closer the cosine similarity is to 1, the closer the directions of the two vectors are and the more similar their semantics are.

[0115] This step ensures that the matching basis is based on directional consistency in the multidimensional semantic space rather than simple numerical proximity.

[0116] The user's current emotion value (for example, inferred by voiceprint recognition and text sentiment analysis) and the urgency factor corresponding to the business scenario are introduced. The two are jointly used to construct the scenario response priority. The influence weight on the final score is controlled by adjusting the factor. The calculation expression is as follows:

[0117] , where It is the scenario response priority, which indicates the priority score adjusted after combining the user's current status and the scenario urgency based on the matching semantic similarity. It is used to give priority to responding to the most urgent or important services. Is the emotion weight coefficient, which adjusts the influence of the user's emotion value on the final priority score. It is generally set through experience or model tuning (such as 0.2-0.5) to ensure that the emotional state has a moderate influence in the scoring mechanism. The current emotion value of the user, ranging from 0 to 1, is determined by the system through the joint evaluation of the voiceprint emotion recognition model and the text sentiment analysis model. For example, the value of a stable emotion is 0.1, and the excitement or anger state can reach 0.9. The larger the value, the more the user needs a quick response. It is the weight coefficient of the scenario urgency, which controls the effect of the urgency of the scenario itself on the priority. It is typically set to 0.3-0.6. For example, it is used to ensure the response weight of the "account freeze release" scenario. It is The risk or urgency score (range: 0-1) for each scenario node is derived from operational experience or historical business data. For example, the urgency of scenarios such as complaints and real-name authentication failures can be set above 0.8, while that of package information inquiries may be set below 0.3.

[0118] This priority not only considers matching accuracy but also introduces dynamic variables such as user experience and task urgency to enhance the contextual sensitivity of the matching logic.

[0119] Adding the time-sensitive factor, that is, the time attenuation function parameter of the scene node relative to the current moment, characterizes the real-time urgency of the scene response, and finally calculates the scene synthesis. The formula is as follows:

[0120] , where It is the final scenario comprehensive score, which is used to select the current optimal business response path among multiple candidate scenarios. It comprehensively considers semantic similarity, user status, scenario importance and time factors. It is the time sensitivity weight coefficient, which controls the influence of the time factor in the scene score. It is generally set at 0.5-1.0 to adjust the penalty mechanism of "the later the response, the more unreasonable it is". is the natural base, Is the current time for The urgency function input variables for each business scenario, such as the time interval between the current interaction and the last similar interaction (unit: minutes), the current business request queue time, and the waiting time after the user raises this type of question;

[0121] The design models the sensitivity to time changes through logical functions to ensure that high-priority but time-sensitive scene nodes are responded to in a timely manner.

[0122] Select the one with the highest The scene node is taken as the final interaction target, and the scene executor is immediately scheduled to trigger the corresponding business process. At the same time, the scoring factors and user response feedback data are synchronously transmitted back to the learning engine for subsequent strategy optimization.

[0123] The above steps improve the accuracy of matching user intent with business scenarios, making them particularly suitable for complex service environments with multiple highly similar candidate scenarios. This scoring mechanism not only considers the basic similarity between the user's current intent and the scenario content, but also incorporates the user's current emotional state, the urgency of the business scenario, and the real-time nature of the request, creating a multi-dimensional, dynamically updated matching scoring standard.

[0124] In actual operation, the system first searches the business scenario database for all candidate business scenario nodes based on the generated user intent vector and evaluates the semantic similarity between them. Initial scoring is based on metrics such as key concept matching and interaction intent coverage. However, semantic matching alone can easily lead to confusion between multiple similar scenarios, so the system proceeds to a further weighting stage.

[0125] During this phase, the system incorporates emotion scores provided by the user emotion recognition module, including indicators of anxiety, anger, confusion, and other emotional states. If a user displays strong negative emotions, such as high-intensity complaints or expressions of dissatisfaction, the system prioritizes business scenarios with shorter response times and fewer processing steps, reducing service pressure and improving response speed.

[0126] At the same time, business scenarios themselves have urgency levels. For example, "unfreezing an account" is generally considered a high-urgency scenario, while "checking points" is considered a low-priority business. The system will consider the scenario's urgency level and the user's emotional state to establish a scenario response priority.

[0127] The system also considers the time sensitivity of the current request. During peak hours or periods of high user access, the system prioritizes processing paths with lower resource loads to avoid performance bottlenecks and ensure service continuity. This time factor dynamically adjusts the scenario score, allowing the same intent to be matched to different scenario nodes at different time periods, thereby achieving dynamic service optimization.

[0128] Through the above-mentioned multi-dimensional comprehensive scoring mechanism, Claim 8 ensures that the system can make efficient and accurate response scenario judgments when faced with complex decisions, significantly improving user satisfaction and system processing stability, and is a key link in supporting the high-precision business execution of intelligent interactive systems.

[0129] The present invention significantly improves the accuracy of user intent recognition by introducing multimodal unified dimension embedding and modal dynamic weighting mechanism. After collecting voice, text and behavioral data, the system suppresses the deviation caused by the imbalance of feature dimensions through standardization and modal confidence-guided attention fusion strategy. Especially in scenarios where user input information is incomplete or environmental interference is significant, the system can automatically identify the quality of the current modality and adjust its weight distribution in the fusion process, so that the final generated user intent representation is more realistic and comprehensive. This mechanism effectively avoids the misjudgment caused by the excessive dominance of a certain modality and significantly reduces the probability of mismatching scenarios.

[0130] This invention combines a reinforcement learning feedback mechanism with a multi-factor scoring model to construct a self-evolving scenario matching system. Based on the similarity between user intent vectors and business scenario vectors, the system further incorporates factors such as sentiment, response urgency, and access timing for weighted correction, thereby achieving a more personalized and dynamic matching process. This structure ensures that the system can accurately identify the most appropriate business scenario node even when faced with complex scenarios and uncertain user expressions, enhancing the automation and intelligence level of the overall business process, and ultimately improving the timeliness and accuracy of service responses.

[0131] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0132] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

[0133] It should be noted that, in this document, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0134] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0135] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0137] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0138] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0139] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0140] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.

Claims

1. AI-based adaptive scene intelligent interaction system, characterized by: It includes multimodal data acquisition module, modal preprocessing module, multimodal embedding coding module, intent fusion and representation module, business scenario matching module, and business execution and reinforcement learning module: Multimodal data acquisition module, which collects multimodal raw data during user interaction, including voice signals, text input, and user behavior trajectories, and simultaneously records the acquisition timestamp; The modal preprocessing module performs preprocessing operations on the collected multimodal raw data. It performs noise reduction and voiceprint emotion analysis on voice signals, performs word segmentation and syntactic analysis on text input, and extracts path features and conducts behavior frequency modeling on user behavior trajectories. The multimodal embedding coding module performs independent embedding coding on the preprocessed multimodal feature data based on an improved dual-channel deep neural network to generate an embedding vector of uniform dimension. The intent fusion and representation module uses the multimodal embedding vector generated by the attention mechanism to construct a user intent representation vector and dynamically monitors the contribution ratio of each modality through a visually adjustable feature weight matrix. The business scenario matching module inputs the user intent representation vector into the pre-built business scenario vector space, identifies the matching degree through cosine similarity calculation, and locates the optimal scenario node; The business execution and reinforcement learning module calls the scenario executor to trigger the business process response based on the identified scenario nodes, and transmits user feedback data back to the feedback reinforcement model in real time, periodically optimizing the intent recognition and scenario matching accuracy through the deep reinforcement learning algorithm.

2. The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that: When collecting voice signals during user interaction, the system further performs spectral analysis on the audio content and extracts key frequency domain features, including fundamental frequency change rate, average frequency band energy density, and short-time energy envelope curve. It also generates multi-dimensional audio tags based on the user's physical environment parameters. In terms of text input collection, real-time conversation context modeling is performed for each user session to construct a text semantic frame including keyword sequences, sentiment tendency scores, intent confidence intervals, etc. For user behavior trajectories, their contact sequence, mouse trajectory angular velocity and dwell time are collected, and a time series behavior model is established through a multi-window sliding average algorithm to provide an accurate data foundation for subsequent embedding processing and behavior intention inversion.

3. The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that: When performing multimodal preprocessing, the voice signal preprocessing uses a voiceprint recognition model based on a convolutional neural network combined with a multi-task learning mechanism to perform multi-label classification prediction on the user's emotional state. The predicted labels include anxiety, anger, confusion, and calmness. The text input component introduces a contextual attention mechanism to capture the characteristics of emotional transitions across sentences and output an emotional transition identifier. Regarding user behavior trajectories, a trajectory clustering algorithm based on a graph neural network is used to cluster and reduce the dimensionality of high-frequency contact areas and output an index of the user's preferred area. All preprocessing results will be normalized before entering the embedding code to ensure the uniform distribution of multimodal inputs and improve the stability and recognition accuracy during subsequent model fusion.

4. The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that: During the embedding encoding process, each type of modal data is processed separately through encoding channels. The voice channel uses a bidirectional GRU encoder with a superimposed positional encoding mechanism, the text channel uses a multi-layer Transformer structure for context modeling, and the user behavior channel introduces a temporal convolution module to extract short-term behavioral change features. When generating an embedding vector of uniform dimension, the projection mapping of each modality embedding vector is dynamically adjusted through a three-channel embedding space alignment mechanism to ensure consistent expressiveness in the shared embedding space. A modality confidence label is also included as an important basis for the subsequent fusion stage.

5. The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that: When performing multimodal vector fusion operations, the attention mechanism uses a stacked multi-head attention network. The first layer focuses on the cross-influence weights between modalities, and the second layer focuses on the temporal dependencies of important features within a single modality. The user intention representation vector output by the fusion module has a dimension of 78; To prevent a single mode from dominating the judgment, a modal balancing factor is introduced to control the modal contribution deviation rate within ±12% after each round of fusion, and the modal weighting coefficient is automatically adjusted through a feedback learning mechanism.

6. The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that: Before generating the user intent representation vector, perform modal feature normalization fusion. This dynamically adjustable feature transformation mechanism improves the stability and recognition accuracy of the model fusion phase. The specific steps are as follows: For each modal embedding vector, the standard deviation normalization method is used to map the eigenvectors of different modalities to a standard normal distribution with a mean of zero and a variance of one in the numerical domain. The calculation expression is as follows: , where The original modality embedding vectors, It is The mean of the embedding vectors generated by the modal embedding vectors in the training sample set, It is The standard deviation of the modal embedding vector, is the normalized modality embedding vectors; Based on the signal-to-noise ratio calculated from historical sampling in the predefined sample set, the expression credibility of each modal embedding vector is quantified. By comparing the energy ratio of useful feature signals and noise interference, the importance weight of the modal embedding vector is calculated. The formula is as follows: , where It is The signal-to-noise ratio of the modal embedding vector, It is The signal-to-noise ratio of the modal embedding vector, is the total number of modes, It is Importance weights of the modality embedding vectors; All normalized embedding vectors are fused using the modality weighted average mechanism to generate a fused comprehensive intent vector. The calculation expression is as follows: , where is the final integrated user intention vector; The obtained user intention comprehensive vector Input into the subsequent attention fusion module to extract deeper semantic dependency features.

7. The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that: When performing business scenario matching operations, the business scenario vector space is composed of 78-dimensional vectors, and each scenario node is generated through prior data training. Its feature vector contains four core features: business type identifier, scenario risk level, user historical satisfaction mean, and contextual behavior similarity; The matching mechanism uses real-time cosine similarity calculation. If the similarity between the current user intention vector and multiple scene nodes is higher than 0.85, the context judgment module is automatically started. The optimal scene node is selected based on the weighted decision of the business frequency priority, user identity label and emotional state score at the current time point.

8. The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that: When calculating scene matching, we introduce an advanced scoring mechanism that integrates user state perception and scene structure characteristics to improve the accuracy of matching results and the real-time nature of business responses. The specific steps are as follows: The user's current intention vector is calculated by cosine similarity with each business scenario node vector predicted and trained to generate an initial semantic similarity score. The formula is as follows: , where It is The similarity score between the business scenario and the user's current intention, is the user intention representation vector, It is Business scenario node vectors, They are vectors and The Euclidean norm of ; The user's current emotion value and the urgency factor corresponding to the business scenario are introduced. The two are jointly used to construct the scenario response priority. The influence weight on the final score is controlled by adjusting the factor. The calculation expression is as follows: , where is the scenario response priority, is the emotion weight coefficient, is the user’s current sentiment value, is the scenario urgency weight coefficient, It is Risk or urgency score of each scenario node; Adding a time-sensitive factor to characterize the real-time urgency of the scene response, the final scene synthesis is calculated as follows: , where is the final scene comprehensive score, is the time sensitivity weight coefficient, is the natural base, Is the current time for The input variables of the urgency function of each business scenario; Select the one with the highest The scene node is taken as the final interaction target, and the scene executor is immediately scheduled to trigger the corresponding business process. At the same time, the scoring factors and user response feedback data are synchronously transmitted back to the learning engine for subsequent strategy optimization.

Images