AI-based adaptive scene intelligent interaction system

Abstract

The application discloses an AI-based adaptive scene intelligent interaction system, and relates to the technical field of intelligent interaction, comprising a multi-modal data acquisition module, a modal preprocessing module, a multi-modal embedding coding module, an intention fusion and representation module, a business scene matching module, and a business execution and reinforcement learning module; the multi-modal data acquisition module acquires multi-modal original data in the user interaction process, including voice signals, text inputs, and user behavior trajectories, and synchronously records acquisition timestamps; the application effectively solves the problem of uneven feature dimension through a multi-modal unified embedding and dynamic weighting mechanism, improves the accuracy of user intention recognition; meanwhile, in combination with a reinforcement learning and a multi-factor scoring model, personalized scene matching and dynamic response are realized, the adaptability and service accuracy of the system in a complex environment are improved, thereby the stability and user experience of the intelligent interaction system are significantly optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent interaction, and particularly relates to an AI-based adaptive scene intelligent interaction system. BACKGROUND

[0002] The AI-based adaptive scene intelligent interaction refers to real-time perception and understanding of multi-modal information (such as voice, text, behavior characteristics, etc.) input by users through artificial intelligence technology, accurate identification of user intent in combination with historical behavior, environmental characteristics and other context factors, and dynamic matching of the most suitable service processing scene. The system analyzes user demand through a deep learning model such as an improved BERT and LSTM network, and continuously optimizes the interaction strategy by using a reinforcement learning mechanism, and finally realizes the whole process of intelligent routing, automatic answering and business handling. The system has self-learning and scene adaptation capabilities, can respond to new business types and sudden demands in real time, and greatly improves service efficiency and user satisfaction.

[0003] The prior art has the following disadvantages: In the existing AI-based adaptive scene intelligent interaction system, the feature dimension imbalance problem in multi-modal information processing is an easily overlooked but destructive challenge. When the system synchronously collects multi-source data such as voice, text and user behavior, due to the differences in data volume, noise sensitivity and feature expression of each modality, the model may be excessively guided by the feature of a certain modality in the intent recognition process, resulting in the neglect of other key information. Especially in the scene where the user's expression is ambiguous, the environment is noisy or the emotion is excited, this deviation is easy to cause wrong judgment, mis-match the business process, and finally may cause that the user's appeal cannot be correctly responded, affect the service quality, and even cause a trust crisis.

[0004] The above information disclosed in the background section is only intended to strengthen the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present application is to provide an AI-based adaptive scene intelligent interaction system, which effectively solves the feature dimension imbalance problem through multi-modal unified embedding and dynamic weighting mechanism, improves the accuracy of user intent recognition, and at the same time realizes personalized scene matching and dynamic response by combining reinforcement learning and multi-factor scoring model, improves the adaptability and service accuracy of the system in complex environment, thereby significantly optimizes the stability and user experience of the intelligent interaction system, to solve the problems in the background technology.

[0006] In order to achieve the above object, the present application provides the following technical scheme: an AI-based adaptive scene intelligent interaction system, comprising a multi-modal data acquisition module, a modal preprocessing module, a multi-modal embedding coding module, an intention fusion and representation module, a business scene matching module, and a business execution and reinforcement learning module:

[0007] The multi-modal data acquisition module acquires multi-modal raw data in the user interaction process, including voice signals, text inputs, and user behavior trajectories, and synchronously records the acquisition timestamps;

[0008] The modal preprocessing module performs preprocessing operations on the acquired multi-modal raw data, wherein the voice signals perform noise reduction and voiceprint emotion analysis, the text inputs perform word segmentation and syntax analysis, and the user behavior trajectories perform path feature extraction and behavior frequency modeling;

[0009] The multi-modal embedding coding module independently embeds and encodes the multi-modal feature data after preprocessing based on an improved dual-channel deep neural network to generate embedding vectors of a unified dimension;

[0010] The intention fusion and representation module fuses the generated multi-modal embedding vectors using an attention mechanism to construct user intention representation vectors, and dynamically monitors the contribution proportion of each modality through a visual adjustable feature weight matrix;

[0011] The business scene matching module inputs the user intention representation vectors into a pre-constructed business scene vector space, identifies the matching degree through cosine similarity calculation, and locates the optimal scene node;

[0012] The business execution and reinforcement learning module, based on the identified scene node, calls a scene executor to trigger a business process response, and feeds back user feedback data to a feedback reinforcement model in real time, periodically optimizes the intention recognition and scene matching accuracy through a deep reinforcement learning algorithm.

[0013] Preferably, when collecting voice signals in the user interaction process, further comprising performing spectral analysis on the audio content and extracting key frequency domain features, including fundamental frequency variation rate, average frequency band energy density, and short-time energy envelope curve, while combining the physical environment parameters of the user to generate multi-dimensional audio labels;

[0014] In terms of text input acquisition, real-time conversation context modeling is performed for each user session to construct a text semantic frame including keyword sequence, sentiment orientation score, and intention confidence interval;

[0015] For user behavior trajectories, the contact sequence, mouse trajectory angular velocity, and dwell time are collected, and a time series behavior model is established through a multi-window moving average algorithm to provide accurate data basis for subsequent embedding processing and behavior intention inversion.

[0016] Preferably, in the execution of multimodal preprocessing, the speech signal preprocessing adopts a voiceprint recognition model based on a convolutional neural network combined with a multi-task learning mechanism to perform multi-label classification prediction on the user emotional state, and the prediction labels include anxiety, anger, confusion and calmness;

[0017] The text input part introduces a context attention mechanism to capture cross-sentence emotion jumps and output emotion transition identifiers. In terms of user behavior trajectory, a trajectory clustering algorithm based on a graph neural network is used to cluster and reduce the dimension of high-frequency touch areas, and output the user preference area index.

[0018] All preprocessing results will be normalized before entering the embedding encoding to ensure the uniform distribution of multi-modal input and improve the stability and recognition accuracy of subsequent model fusion.

[0019] Preferably, in the embedding encoding process, each type of modal data is processed through an encoding channel. The speech channel uses a bidirectional GRU encoder with position encoding mechanism, the text channel uses a multi-layer Transformer structure for context modeling, and the user behavior channel introduces a time convolution module to extract short-period behavior change features.

[0020] When generating embedding vectors of a unified dimension, a three-channel embedding space alignment mechanism is used to dynamically adjust the projection mapping of each modal embedding vector, so that it has consistent expression ability in the shared embedding space, and is accompanied by a modal confidence label, which is an important basis for the subsequent fusion stage.

[0021] Preferably, in the execution of multi-modal vector fusion operation, the attention mechanism adopts a stacked multi-head attention network, the first layer focuses on the cross-influence weight between modalities, and the second layer focuses on the time sequence dependency of important features within a single modality.

[0022] The user intent representation vector output by the fusion module has a dimension of 78.

[0023] To prevent a single modality from dominating the judgment, a modal balance factor is introduced to control the modal contribution deviation rate within ±12% after each round of fusion, and the modal weighting coefficient is automatically adjusted through a feedback learning mechanism.

[0024] Preferably, before generating the user intent representation vector, modal feature normalization fusion is performed to improve the stability and recognition accuracy of the model fusion stage through a dynamically adjustable feature transformation mechanism. The specific steps are as follows:

[0025] For each modal embedding vector, the standard deviation normalization method is used to map the feature vectors of different modalities to a standard normal distribution with a mean of zero and a variance of one in the numerical domain. The calculation expression is as follows:

[0026] , wherein, is the original first modal embedding vector, is the first modal embedding vector, is the mean of the embedding vectors generated by the first modal embedding vector in the training sample set, is the standard deviation of the first modal embedding vector,

[0027] According to the signal-to-noise ratio calculated according to the historical sampling in the predefined sample set, the expression credibility of each modal embedding vector is quantified, and the importance weight of the modal embedding vector is calculated by comparing the energy proportion of the useful feature signal and the noise interference, and the formula is as follows:

[0028] , wherein, is the signal-to-noise ratio of the first modal embedding vector, is the signal-to-noise ratio of the first modal embedding vector, is the total number of modes, is the importance weight of the first modal embedding vector;

[0029] The modal weighted average mechanism is used to fuse all the standardized embedding vectors to generate a fused comprehensive intention vector, and the calculation expression is as follows:

[0030] , wherein, is the user intention comprehensive vector finally fused;

[0031] The user intention comprehensive vector obtained is input into a subsequent attention fusion module to extract deeper semantic dependency features.

[0032] Preferably, when performing a business scenario matching operation, the business scenario vector space is composed of 78-dimensional vectors, each scenario node is generated by prior data training, and the feature vector thereof contains four types of core features: business type identifier, scene risk level, user historical satisfaction mean, and context behavior similarity.

[0033] The matching mechanism adopts real-time cosine similarity calculation, and if the similarity between the current user intention vector and multiple scene nodes is higher than 0.85, the context judgment module is automatically started, the optimal scene node is selected according to the business frequency priority at the current time point, the user identity label and the emotion state score weighted decision.

[0034] Preferably, in the process of scene matching degree calculation, a set of advanced scoring mechanism integrating user state awareness and scene structure features is introduced to improve the accuracy of matching results and the real-time performance of business response. The specific steps are as follows:

[0035] The cosine similarity between the current intention vector of the user and each business scene node vector trained is calculated to generate an initial semantic similarity score, and the formula is as follows:

[0036] , wherein, is the similarity score between the i-th business scene and the current intention of the user, is the similarity score between the i-th business scene and the current intention of the user, is the user intention representation vector, is the i-th business scene node vector, is the i-th business scene node vector, is the Euclidean norm of the vector and ; and

[0037] The current emotional value of the user and the urgency factor corresponding to the business scene are introduced to jointly participate in constructing the scene response priority. The influence weight of the final score is controlled by adjusting the factor, and the calculation expression is as follows:

[0038] , wherein, is the scene response priority, is the emotional weight coefficient, is the current emotional value of the user, is the scene urgency weight coefficient, is the risk or urgency score of the i-th scene node; The time sensitivity factor is added to depict the real-time urgency of the scene response, and the final scene comprehensive score is calculated, and the formula is as follows:

[0039]

[0040] , wherein, is the final scene comprehensive score, is the time sensitivity weight coefficient, is the natural base, is the urgency function input variable of the current time to the i-th business scene; The scene node with the highest score is selected as the final interaction target, and the scene executor is immediately dispatched to trigger the corresponding business process. At the same time, the factors of this scoring and the user response feedback data are synchronously returned to the learning engine for subsequent strategy optimization.

[0041] The scene node with the highest score is selected as the final interaction target, and the scene executor is immediately dispatched to trigger the corresponding business process. At the same time, the factors of this scoring and the user response feedback data are synchronously returned to the learning engine for subsequent strategy optimization.

[0042] ​​In the above technical solution, the present application provides technical effects and advantages:

[0043] The present application introduces a multi-modal unified dimension embedding and a modal dynamic weighting mechanism, which significantly improves the accuracy of user intent recognition. After collecting voice, text and behavior data, the system uses a standardized and modal confidence guided attention fusion strategy to suppress the bias caused by uneven feature dimensions. Especially in the scene where the user input information is incomplete or the environmental interference is significant, the system can automatically identify the current modal quality and adjust the weight distribution in the fusion process, so that the finally generated user intent representation is more real and comprehensive. This mechanism effectively avoids the judgment error caused by the dominance of a certain modal, and significantly reduces the probability of mis-matching scenarios.

[0044] The present application combines a reinforcement learning feedback mechanism and a multi-factor scoring model to build a self-evolving scene matching system. Based on the similarity between the user intent vector and the business scene vector, the system further introduces emotion value, response urgency, access timing and other factors for weighted correction, so as to realize a more personalized and dynamic matching process. This structure ensures that the system can accurately identify the most suitable business scene node when facing complex scenes and uncertain user expressions, improves the automation and intelligence level of the overall business process, and ultimately improves the timeliness and accuracy of service response. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments or prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0046] Figure 1 The module schematic diagram of the AI-based adaptive scene intelligent interaction system of the present application. DETAILED DESCRIPTION

[0047] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these example implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects to those skilled in the art.

[0048] The present application provides an AI-based adaptive scene intelligent interaction system as shown in Figure 1 The AI-based adaptive scene intelligent interaction system includes a multi-modal data acquisition module, a modal preprocessing module, a multi-modal embedding coding module, an intent fusion and representation module, a business scene matching module, and a business execution and reinforcement learning module.

[0049] A multi-modal data acquisition module acquires multi-modal raw data in the user interaction process, including voice signals, text inputs, and user behavior trajectories, and synchronously records acquisition timestamps;

[0050] When acquiring voice signals in the user interaction process, further including performing spectral analysis on the audio content and extracting key frequency domain features, including fundamental frequency variation rate, average frequency band energy density, short-time energy envelope curve, while combining with physical environment parameters (such as background noise decibel value, sound source direction angle) of the user to generate multi-dimensional audio labels;

[0051] In terms of text input acquisition, real-time session context modeling is performed for each user session to build a text semantic frame including keyword sequence, sentiment orientation score, and intent confidence interval;

[0052] For user behavior trajectories, touch sequence, mouse trajectory angular velocity, and dwell time are collected, and a time series behavior model is established through a multi-window moving average algorithm to provide accurate data basis for subsequent embedding processing and behavior intent inversion.

[0053] The above steps enhance the integrity and context relevance of multi-modal data acquisition during user interaction, and through high-dimensional multi-source input, ensure that the subsequent AI processing module obtains accurate and rich data basis, solving the problem of recognition bias caused by data isolation and feature missing in traditional systems. In actual systems, three types of capabilities are required to implement this step: high-fidelity data capture, multi-dimensional label extraction, and inter-modal time synchronization.

[0054] First, the processing of voice signals is not limited to raw waveform capture, but should be based on short-time Fourier transform (STFT) to calculate the spectrogram and extract features such as fundamental frequency variation rate, frequency band energy density, and voiceprint energy concentration. These features can reveal the user's emotions, speech speed, and tone changes, helping AI to identify voice emotional states such as anxiety and anger.

[0055] Secondly, in the text input layer, word segmentation and part-of-speech tagging are performed through natural language processing tools (such as HanLP, spaCy), while semantic vectors are constructed, sentiment orientation (such as anger, doubt, negative sentiment) is recognized, and keywords are extracted. If the user simultaneously issues multiple statements, an emotion jump detection mechanism can be introduced to identify potential demand mutations.

[0056] Finally, user behavior trajectories need to be sampled from touch trajectory, click hotspot distribution, mouse movement angular velocity, and other dimensions to establish a sliding window behavior model. Each operation time point is time-stamped to ensure alignment with voice and text data. These trajectory patterns can reveal whether the user is confused (frequent clicks), hesitant (long hover), and other implicit behavior intentions.

[0057] Through the complete acquisition and annotation of the above three modalities, the system has a strong foundation for raw data input, laying a solid foundation for subsequent high-precision intelligent recognition.

[0058] The modal preprocessing module performs preprocessing operations on the collected multimodal raw data. Specifically, it performs noise reduction and voiceprint emotion analysis on speech signals, word segmentation and syntactic analysis on text input, and path feature extraction and behavior frequency modeling on user behavior trajectories.

[0059] When performing multimodal preprocessing, the speech signal preprocessing adopts a voiceprint recognition model based on convolutional neural network combined with a multi-task learning mechanism to perform multi-label classification prediction of the user's emotional state. The predicted labels include four categories: anxiety, anger, confusion, and calmness.

[0060] The text input section introduces a contextual attention mechanism to capture the emotional shifts across sentences and output an emotion transfer identifier. In terms of user behavior trajectory, a trajectory clustering algorithm based on graph neural networks is used to cluster and reduce the dimensionality of high-frequency touchpoint areas and output a user preference area index.

[0061] All preprocessing results will be normalized before entering the embedding encoding process to ensure a uniform distribution of multimodal inputs and improve the stability and recognition accuracy during subsequent model fusion.

[0062] The steps described above enhance the intelligence and robustness of the data preprocessing stage, ensuring that multimodal input data possesses semantic richness, emotional readability, and clear behavioral structure before entering the deep model. Traditional AI systems often neglect preprocessing, leading to limited subsequent recognition performance. This method effectively addresses this shortcoming by introducing emotion recognition, multi-task learning, and graph-structured behavior modeling.

[0063] First, the speech component uses a CNN model to extract parameters such as Mel-frequency cepstral coefficients (MFCCs) and then incorporates them into a multi-task learning framework for emotion label classification. Emotion classification includes four types: anxiety, anger, confusion, and calmness. The multi-label results are trained using a cross-entropy loss function, which helps subsequent modules understand the user's current mood.

[0064] Secondly, the text input preprocessing uses the BERT+CRF model to perform entity recognition and sentiment analysis on the text. The context attention mechanism tracks semantic jumps between sentences and detects complex intentions such as "I want to cancel but I'm worried about the cost," providing sentiment and conflict state annotations for semantic fusion.

[0065] Behavioral trajectory data is used to construct user behavior graphs (e.g., click nodes are graph nodes and paths are edges), and then clustered using graph neural networks (GNNs) to identify high-frequency click areas, operation backtracking behaviors, and interface adaptation difficulties, forming structured behavioral feature vectors.

[0066] The above processing results will be uniformly normalized (such as Z-Score or Min-Max method) to make different modalities consistent in scale, and enter the next step of embedding modeling process. This step effectively solves the problem of uneven expression caused by modal difference.

[0067] The multi-modal embedding coding module independently encodes the pre-processed multi-modal feature data based on an improved dual-channel deep neural network to generate embedding vectors of uniform dimension, so as to avoid the dominant effect of the expression of a certain modality in the feature space.

[0068] During the embedding coding process, each type of modality data is processed through the coding channel, wherein the speech channel adopts a bidirectional GRU encoder and superimposes a position encoding mechanism, the text channel adopts a multi-layer Transformer structure for context modeling, and the user behavior channel introduces a time convolution module to extract short-period behavior change features.

[0069] When generating embedding vectors of uniform dimension, a three-channel embedding space alignment mechanism is used to dynamically adjust the projection mapping of each modality embedding vector, so that it has consistent expression ability in the shared embedding space, and is accompanied by a modality confidence label as an important basis for the subsequent fusion stage.

[0070] The purpose of the above steps is to build a professional coding path for multi-modal data, so that the speech, text, behavior and other modal data have consistent expression ability in the unified embedding space, preventing the risk of any modality dominating the final model judgment. Its implementation relies on three mechanisms: modality separation coding, cross-modal projection, and confidence labeling.

[0071] First, the speech channel adopts a bidirectional GRU (BiGRU) network, which can capture both forward and backward features of speech data. And before encoding, position encoding (such as Sinusoidal Position Encoding) is embedded to improve the ability to understand the order of speech. This method can enhance the model's ability to recognize emotional changes and intent shifts in long speech segments.

[0072] Second, the text channel adopts a multi-layer Transformer structure (which can contain 6 or 12 layers) to model self-attention for input semantic segments and capture language logic such as subject-predicate-object structure, negative intent, emotional questions, etc. Through residual connection and layer normalization mechanism, the original syntax information is preserved to improve the accuracy of semantic encoding.

[0073] The user behavior channel introduces a one-dimensional time convolution network (TCN) that can extract time series features such as click frequency trends and reaction speed differences within a short period of time, generating high-dimensional behavior embedding vectors.

[0074] After the three types of encoding outputs, enter a modal vector alignment module, map each modality to a shared 78-dimensional semantic space through a learnable linear projection matrix, and add modal confidence (such as speech signal quality score, text integrity, etc.) to provide adjustable and reliable indicators for the fusion stage.

[0075] The intention fusion and representation module uses attention mechanism to fuse the generated multi-modal embedding vectors, constructs user intention representation vectors, and dynamically monitors the contribution proportion of each modality through a visual adjustable feature weight matrix.

[0076] When performing multi-modal vector fusion operations, the attention mechanism uses a stacked multi-head attention network, the first layer focuses on the cross-influence weight between modalities, and the second layer focuses on the time sequence dependence of important features within a single modality.

[0077] The user intention representation vector output by the fusion module has a dimension of 78, and each dimension represents a specific semantic or behavior dimension, such as behavior tension index, language emotion fluctuation rate, semantic complexity score, etc.

[0078] To prevent a single modality from dominating the judgment, a modal balance factor is introduced to control the modal contribution deviation rate within ±12% after each round of fusion, and automatically adjust the modal weighting coefficients through a feedback learning mechanism.

[0079] The role of the above steps is to fuse multiple modal embedding vectors into semantic-consistent and behavior-sensitive user intention representation vectors, and overcome the problem of a single modality dominating the recognition results in traditional models by dynamically controlling the modal contribution proportion. The fusion process is based on a multi-layer attention mechanism and is self-adaptively regulated by a modal balance factor and a strategy learning mechanism.

[0080] In terms of specific implementation, the fusion module includes two layers of attention networks. The first layer is an inter-modal attention mechanism that learns the mutual correlation between modalities, such as whether "expressing anxious emotions in user speech" is homologous to "quickly clicking to exit the page". This mechanism generates an inter-modal attention weight matrix by calculating the correlation coefficient between inter-modal embedding vectors.

[0081] The second layer is an intra-modal attention mechanism that deeply excavates feature dimensions with high semantic value within a single modality. For example, in the text modality, the attention weight of identifying keywords such as "cancel", "complaint", and "unable to use" is much higher than that of ordinary auxiliary words. This mechanism calculates the weight distribution of different positions within a single modality through a multi-head self-attention structure, and preferentially transmits the high-weight part to the fusion output.

[0082] To prevent the modal dominance problem, the system introduces a "modal deviation rate" evaluation mechanism. In the fused user intent vector, if a modal weight exceeds 60% of the overall weight, the modal balance adjustment process is triggered, and the attention weight of the modal is adjusted to within ±12% of the average level. This deviation control strategy ensures that all modes are expressed fairly, improving the robustness of the fusion vector.

[0083] After the fusion vector is output, it enters the strategy optimization module, which periodically updates the fusion strategy based on actual business feedback (user satisfaction, whether to transfer to human, etc.) through deep reinforcement learning methods such as DQN or PPO. In this way, the fusion mechanism can evolve continuously according to real scenarios, ensuring that the intent vector always maintains strong recognition and generalization capabilities for business goals.

[0084] Before generating the user intent representation vector, modal feature normalization fusion is performed to solve the structural inconsistency between multi-modal inputs in terms of distribution density, scale size, and numerical domain, and to improve the stability and recognition accuracy of the model fusion stage. The specific steps are as follows:

[0085] For each modal embedding vector, the standard deviation normalization method is used to map the feature vectors of different modalities to a standard normal distribution with a mean of zero and a variance of one in the numerical domain, thereby eliminating the dominance of the original embedding vector in absolute numerical values. The calculation expression is as follows:

[0086] , where is the original th modal embedding vector, representing the vector output of a certain modality (such as speech, text, behavior) after processing by the embedding encoding module, with a dimension consistent with the system's intent expression dimension (such as 78 dimensions), is the mean of the th modal embedding vector in the training sample set, representing the center position of the modal vector distribution, which is a historical statistic, is the standard deviation of the th modal embedding vector, measuring the dispersion of the modal feature distribution; the larger the value, the stronger the volatility of the modal in the feature dimension, is the normalized th modal embedding vector, with a standard normal distribution with a mean of 0 and a standard deviation of 1, used to eliminate the interference of scale differences between modalities on model judgment;

[0087] This processing ensures that in subsequent weighted calculations, the influence of each modality comes from semantic expression rather than numerical size.

[0088] Based on the signal-to-noise ratio calculated from historical sampling in a predefined sample set, the credibility of each modality embedding vector is quantified. By comparing the energy proportions of useful feature signals and noise interference, the importance weight of the modality embedding vector is calculated, as shown in the following formula:

[0089] In the formula, It is the first The signal-to-noise ratio (SNR) of a modal embedding vector is a ratio that measures the effectiveness of the modal signal relative to the degree of interference. For example, for speech modalities, it can be estimated based on the energy of the MFCC feature signal and background noise; for text modalities, it can be estimated from the word vector confidence interval bias. It is the first The signal-to-noise ratio (SNR) of a modal embedding vector is an indicator that measures the ratio of useful information (signal) to interference information (noise) in a modal data. It is the total number of modes. It is the first The importance weights of each modality embedding vector are used in the subsequent multimodal weighted fusion stage and are key parameters for balancing the forces of each input source in the decision-making process.

[0090] This weight reflects the relative reliability of each modality in semantic expression under the current environmental conditions, thereby improving the model's adaptive capability in the case of multi-source heterogeneous input.

[0091] The modality-weighted averaging mechanism is used to fuse all standardized embedding vectors to generate a fused comprehensive intent vector. The calculation expression is as follows:

[0092] In the formula, It is the final fused comprehensive user intent vector, with the same dimensions as the single-modal embedding (e.g., 78 dimensions), which integrates the standardized semantic features of all modalities and takes into account the differences in information contribution between modalities;

[0093] This operation ensures that the model can fully balance the asymmetry of information between different modalities when comprehensively expressing the user's intent.

[0094] The obtained user intent composite vector The input is fed into the subsequent attention fusion module to extract deeper semantic dependency features, and finally outputs an accurate and stable user intent representation vector for use by the scene matching engine.

[0095] The above steps, starting from the quality of modal input, effectively fuse data from various modalities through standardization and weight adjustment mechanisms to avoid any single modality from dominating and misleading the recognition process. The ultimate goal is to output a comprehensive and balanced user intent representation vector that accurately reflects the user's true intent in specific business interaction scenarios.

[0096] In the implementation process, first, modal data of different sources such as voice, text and behavior are standardized. The main purpose of standardization is to eliminate the order of magnitude difference caused by different sources and dimensions of various data, and to ensure that the system evaluates each modal input fairly. Through this unified scale conversion, the system avoids bias to the modal with larger numerical amplitude, thereby providing a stable data basis for subsequent processing.

[0097] On this basis, the system also introduces a modal signal quality evaluation mechanism, mainly based on historical noise level, stability, data missing rate and other indicators of each modal, to calculate a relative signal-to-noise ratio score for each modal. Subsequently, according to the score, set the weight of each modal in the final fusion vector. For example, in some interaction process, if the system detects that the user's speech is clear but the behavior data is too sparse, it will give the voice modal a higher influence weight to ensure recognition accuracy.

[0098] After completing the weighted fusion, the system combines all modal inputs into a unified fusion vector, and takes the fusion vector as the input of the subsequent attention mechanism. The attention mechanism further strengthens the semantic features of the fused vector, ensuring that in the actual semantic space, the fusion result not only retains the information coordination relationship between the modes, but also has sufficient context expression ability.

[0099] Through the above process, a multi-modal fusion mechanism that takes into account the balance of modal expression, semantic clarity and data reliability is realized, providing strong perception layer support for intelligent interaction systems.

[0100] The business scenario matching module inputs the user intent representation vector into the pre-constructed business scenario vector space, and identifies the matching degree through cosine similarity calculation to locate the optimal scenario node.

[0101] When performing the business scenario matching operation, the business scenario vector space is composed of 78-dimensional vectors, and each scenario node is generated by prior data training, and its feature vector contains four types of core features: business type identifier, scenario risk level, user historical satisfaction average, and context behavior similarity.

[0102] The matching mechanism uses real-time cosine similarity calculation. If the similarity of the current user intent vector to multiple scenario nodes is higher than 0.85, the context judgment module is automatically started, and the optimal scenario node is selected according to the weighted decision of the business frequency priority at the current time point, user identity label and emotion state score.

[0103] The function of the above step is to accurately match user intent by constructing a structured business scenario vector space and combining context factors, ensuring that the final scenario selection meets user needs and conforms to the current business environment characteristics. In actual deployment, this step greatly optimizes the matching error rate caused by static rules in traditional systems, especially in multi-service integration platforms.

[0104] The construction of the business scenario vector space is based on actual business scenario statistical data. Each node represents a type of business scenario (such as "broadband renewal" and "traffic anomaly diagnosis"). Each node corresponds to a 78-dimensional feature vector. This vector includes business identifiers (such as BID-001), scenario risk levels (low, medium, and high), historical satisfaction averages (0-1 decimal), and behavior similarity reference values. Each dimension has a clear physical meaning, ensuring that the model is interpretable during the matching process.

[0105] After generating the user intent representation vector, the system performs vector cosine similarity calculation on all scenario nodes and outputs a ranking list. If the similarity of the top 3 scenario nodes is greater than 0.85, the system will enter the context fine matching stage.

[0106] The context matching stage evaluates the following three types of weight factors:

[0107] The frequency of business requests at the current time point: such as prioritizing complaint-type businesses during the evening peak period;

[0108] User identity tags: such as higher priority for VIP customers;

[0109] User current emotional state score: such as users with extreme emotions, prefer to choose scenarios with high processing efficiency.

[0110] The above factors will be re-ranked through a weighted scoring mechanism, and the optimal scenario node will be selected to drive the executor response. This mechanism takes into account both scenario semantic similarity and environmental adaptability, effectively improving matching accuracy and user satisfaction.

[0111] Business execution and reinforcement learning module, based on the identified scenario node, calls the scenario executor to trigger the business process response, and returns the user feedback data to the feedback reinforcement model in real time, periodically optimizes the intent recognition and scenario matching accuracy through deep reinforcement learning algorithm;

[0112] When calculating the scenario matching degree, a high-level scoring mechanism that combines user state awareness and scenario structure characteristics is introduced to address the problem of not being able to dynamically consider context factors (such as emotional fluctuations, scenario stress levels, and time sensitivity) in traditional matching degree models, improving the accuracy of matching results and the real-time nature of business responses. The specific steps are as follows:

[0113] The current intention vector of the user is calculated with the cosine similarity of each business scenario node vector predicted by training, to generate an initial semantic similarity score, as follows:

[0114] , wherein, is the similarity score between the th business scenario and the current intention of the user, with a value range of 0 to 1, is the user intention representation vector, which is a 78-dimensional vector generated by the system through preprocessing, embedding, and fusion of the user's input multi-modal data (voice, text, behavior, etc.). Each dimension represents a certain feature expression of the user's current intention (such as semantic intensity, behavior characteristics, or emotional index, etc.), which is an accurate encoding of the current interaction semantics by the system, is the th business scenario node vector, which is obtained by pre-training and represents the known business process scenarios of the system, such as "package change" and "network problem complaint". Each scenario vector is also 78-dimensional, with a consistent dimension distribution as , containing information such as business type label, context preference weight, and historical user behavior vector, are the Euclidean norms of vectors and , respectively, used to calculate the standardized cosine similarity to avoid errors caused by differences in vector length. The closer the cosine similarity is to 1, the closer the direction of the two vectors is, and the more similar the semantics are;

[0115] This step ensures that the matching is based on the consistency of direction in the multi-dimensional semantic space rather than simply the numerical proximity.

[0116] The user's current emotional value (e.g. inferred by voiceprint recognition and text sentiment analysis) and the urgency factor corresponding to the business scenario are introduced, both of which participate in building the scenario response priority. The influence weight of the adjustment factor on the final score is controlled through adjustment, and the calculation expression is as follows:

[0117] , wherein, is the scenario response priority, which represents the adjusted priority score after considering the user's current state and the urgency of the scenario based on the matching semantic similarity, and is used to respond to the most urgent or most important service, is the emotional weight coefficient, which adjusts the influence of the user's emotional value on the final priority score, and is generally set through experience or model tuning (such as 0.2-0.5), to ensure that the emotional state has a moderate influence on the scoring mechanism, is the current emotional value of the user, with a value range of 0-1, which is evaluated by the system through a combination of voiceprint emotion recognition model and text sentiment analysis model; for example, the emotional stability value is 0.1, and the excited or angry state can reach 0.9. The larger the value, the more the user is in a state that requires a faster response, is the scene emergency degree weight coefficient, which controls the role of the scene itself in the emergency degree in the priority, and is typically set to 0.3-0.6, for example, for the response weight of the "account unfreezing" scene, is the risk or emergency degree score (range: 0-1) of the th scene node, which is obtained from operation experience or historical business data statistics, for example, the emergency degree of complaint type and real-name authentication failure type scenes can be set to 0.8 or higher, while the emergency degree of the package information query type can be lower than 0.3;

[0118] The priority takes into account the matching accuracy, introduces dynamic variables such as user experience and task urgency, and enhances the context sensitivity of the matching logic.

[0119] The time-sensitive factor is added, i.e. the time decay function parameter of the scene node relative to the current time, which describes the real-time urgency of the scene response, and the final scene comprehensive score is calculated, the formula is as follows:

[0120] , in the formula, is the final scene comprehensive score, which is used to select the current optimal business response path among multiple candidate scenes, considering the semantic similarity, user state, scene importance and time factor, is the time sensitivity weight coefficient, which controls the influence degree of the time factor in the scene score, and is generally set to 0.5-1.0, which is used to adjust the punishment mechanism of "the later the response, the more unreasonable it is", is the natural base, is the urgency function input variable of the current time for the th business scene, for example: the time interval (unit: minutes) between the current interaction and the last similar interaction, the queuing time of the current business request, and the waiting time of the user after raising the problem;

[0121] This design models the sensitivity of time change through a logical function, ensuring that high-priority but time-critical scene nodes are responded to in a timely manner.

[0122] The scene node with the highest is selected as the final interaction target, and the scene executor is immediately dispatched to trigger the corresponding business process, and the score factors of this time and the user response feedback data are simultaneously returned to the learning engine for subsequent strategy optimization.

[0123] The above steps improve the matching accuracy between user intent and business scenarios, especially in complex service environments with multiple high-similarity candidate scenarios. This scoring mechanism not only considers the basic similarity between the user's current intent and the scenario content, but also combines the user's current emotional state, the urgency level of the business scenario, and the real-time factors of the request time to build a multi-dimensional and dynamically updated matching scoring standard.

[0124] In actual operation, the system first retrieves all candidate business scenario nodes in the business scenario database according to the generated user intent vector and evaluates the semantic similarity between them. The preliminary scoring criteria include key concept matching degree, interactive intent coverage, etc. However, relying solely on semantic matching can easily cause confusion among multiple similar scenarios, so the system enters the further weighting stage.

[0125] In this stage, the system introduces emotional scoring provided by the user emotion recognition module, such as anxiety, anger, confusion, and other emotional state indicators. If the user shows strong negative emotions, such as high-intensity complaints or expressions of dissatisfaction, the system will preferentially select business scenarios with short response times and fewer processing chains to reduce service pressure and improve response speed.

[0126] At the same time, the business scenario itself also has an urgency setting, for example, "account unfreezing" is usually considered a high-urgency scenario, while "points inquiry" is a low-priority business. The system will consider the urgency level of the scenario and the user's emotional state to build the scenario response priority.

[0127] In addition, the system also considers the time sensitivity factor of the current request. During peak business hours or periods of high user access frequency, the system will preferentially select processing paths with lower resource load to avoid performance bottlenecks and ensure service continuity. This time factor dynamically adjusts the scenario scoring, so that the same intent may match different scenario nodes at different times, thus achieving dynamic optimization of services.

[0128] Through the above multi-dimensional comprehensive scoring mechanism, the system can make efficient and accurate response scenario judgments when faced with complex decisions, significantly improving user satisfaction and system processing stability, and is a key link to support high-precision business execution of intelligent interaction systems.

[0129] The application significantly improves the accuracy of user intent recognition by introducing a multi-modal unified dimension embedding and a modal dynamic weighting mechanism. After collecting voice, text and behavior data, the system suppresses the bias caused by uneven feature dimensions through a standardized and modal confidence guided attention fusion strategy. Especially in scenarios where user input information is incomplete or environmental interference is significant, the system can automatically identify the current modal quality and adjust its weight distribution in the fusion process, so that the final generated user intent representation is more real and comprehensive. This mechanism effectively avoids the judgment errors caused by the dominance of a certain modal, significantly reducing the probability of mis-matching scenarios.

[0130] The application combines a reinforcement learning feedback mechanism and a multi-factor scoring model to build a self-evolving scene matching system. Based on the similarity between the user intent vector and the business scene vector, the system further introduces emotion value, response urgency, access timing and other factors for weighted correction, thereby realizing a more personalized and dynamic matching process. This structure ensures that the system can accurately identify the most suitable business scene node when faced with complex scenarios and uncertain user expressions, improving the automation and intelligence level of the overall business process, and ultimately improving the timeliness and accuracy of service response.

[0131] The above formulas are dimensionless numerical calculations, and the formulas are obtained by software simulation of a large amount of data to obtain a formula of the most recent real situation. The preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0132] The above only describes some exemplary embodiments of the application by way of illustration, and it is not necessary to modify the described embodiments in various ways without deviating from the spirit and scope of the application for ordinary skilled in the art. Therefore, the above figures and descriptions are illustrative in nature and should not be understood as limiting the scope of protection of the claims of the application.

[0133] It should be noted that in this paper, if there are relationship terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the sentence "includes a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0134] It should be understood that the size of the sequence number of the above processes does not mean the order of execution in various embodiments of the present application, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0135] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0137] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment of the present application according to actual needs.

[0138] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0139] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0140] The above only describes some exemplary embodiments of the present application by way of illustration, and it is needless to say that those skilled in the art can modify the described embodiments in various ways without deviating from the spirit and scope of the present application. Therefore, the above figures and descriptions are illustrative in nature and should not be understood as limiting the scope of the claims of the present application.

Claims

1. An AI-based adaptive scene intelligent interaction system, characterized in that, The system comprises a multi-modal data acquisition module, a modal pre-processing module, a multi-modal embedding coding module, an intention fusion and representation module, a business scenario matching module, and a business execution and reinforcement learning module. The multi-modal data acquisition module acquires multi-modal raw data in the user interaction process, including voice signals, text inputs, and user behavior trajectories, and synchronously records the acquisition timestamps. The modal pre-processing module performs pre-processing operations on the acquired multi-modal raw data, including noise reduction and voiceprint emotion analysis for voice signals, word segmentation and syntax analysis for text inputs, and path feature extraction and behavior frequency modeling for user behavior trajectories. The multi-modal embedding coding module independently embeds and encodes the pre-processed multi-modal feature data based on an improved dual-channel deep neural network to generate embedding vectors of a unified dimension. The intention fusion and representation module fuses the generated multi-modal embedding vectors using an attention mechanism to construct user intention representation vectors and dynamically monitors the contribution proportion of each modality through a visual adjustable feature weight matrix. The business scenario matching module inputs the user intention representation vectors into a pre-constructed business scenario vector space, identifies the matching degree through cosine similarity calculation, and locates the optimal scenario node. The business execution and reinforcement learning module calls a scenario executor to trigger a business process response based on the identified scenario node, and feeds back user feedback data to a feedback reinforcement model in real time to periodically optimize the intention recognition and scenario matching accuracy through a deep reinforcement learning algorithm. Before generating the user intention representation vectors, modal feature normalization fusion is performed through a dynamic adjustable feature transformation mechanism to improve the stability and recognition accuracy of the model fusion stage. The specific steps are as follows: For each modal embedding vector, the standard deviation normalization method is used to map the feature vectors of different modalities to a standard normal distribution with a mean of zero and a variance of one in the numerical domain. The calculation expression is as follows: In the formula, It is the original first A modal embedding vector, It is the first The mean of the embedding vectors generated by each modality embedding vector in the training sample set. It is the first The standard deviation of each modality embedding vector It is the normalized first One modal embedding vector; According to the signal-to-noise ratio calculated from the historical sampling in the pre-defined sample set, the expression credibility of each modal embedding vector is quantified. By comparing the energy proportion of useful feature signals and noise interference, the importance weight of the modal embedding vector is calculated. The formula is as follows: wherein, is a signal-to-noise ratio of the th modality embedding vector, is a signal-to-noise ratio of the th modality embedding vector, is a total number of modalities, is an importance weight of the th modality embedding vector; The modal weighted average mechanism is used to fuse all the normalized embedding vectors to generate a comprehensive intention vector. The calculation expression is as follows: , wherein, is the final fusion user intent comprehensive vector; The obtained user intention synthesis vector is input to a subsequent attention fusion module to extract deeper semantic dependency features. 2.The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that, When collecting voice signals in the user interaction process, further including performing spectral analysis on the audio content and extracting key frequency domain features, including fundamental frequency variation rate, average frequency band energy density, and short-time energy envelope curve, while combining with the physical environment parameters of the user to generate multi-dimensional audio labels. In terms of text input acquisition, real-time session context modeling is performed for each user session to construct text semantic frames including keyword sequences, sentiment orientation scores, and intention confidence intervals. For user behavior trajectories, the contact sequence, mouse trajectory angular velocity, and dwell time are collected, and a time series behavior model is established through a multi-window sliding average algorithm to provide accurate data basis for subsequent embedding processing and behavior intention inversion. 3.The AI-based adaptive scene intelligent interaction system according to claim 1, characterized in that, In the execution of multi-modal preprocessing, the speech signal preprocessing adopts a voiceprint recognition model based on a convolutional neural network combined with a multi-task learning mechanism to perform multi-label classification prediction on the user's emotional state. The predicted labels include anxiety, anger, confusion, and calmness. The text input part introduces a context attention mechanism to capture cross-sentence emotional jumps and output emotion transition identifiers. In terms of user behavior trajectory, a graph neural network-based trajectory clustering algorithm is used to cluster and reduce the dimensionality of high-frequency touch areas and output user preference area indexes. All preprocessing results will be normalized before entering the embedding encoding to ensure the uniform distribution of multi-modal inputs and improve the stability and recognition accuracy of subsequent model fusion. 4.The AI-based adaptive scene intelligent interaction system according to claim 1, wherein, In the embedding encoding process, each type of modal data is processed through an encoding channel. The speech channel uses a bidirectional GRU encoder with position encoding mechanism, the text channel uses a multi-layer Transformer structure for context modeling, and the user behavior channel introduces a temporal convolution module to extract short-period behavior change features. In generating uniform dimension embedding vectors, a three-channel embedding space alignment mechanism is used to dynamically adjust the projection mapping of each modal embedding vector to ensure consistent expression ability in the shared embedding space. The modal confidence label is also attached as an important basis for the subsequent fusion stage. 5.The AI-based adaptive scene intelligent interaction system according to claim 1, wherein, In the multi-modal vector fusion operation, the attention mechanism uses a stacked multi-head attention network. The first layer focuses on the cross-influence weight between modalities, and the second layer focuses on the temporal dependence of important features within a single modality. The user intent representation vector output by the fusion module has a dimension of 78. To prevent a single modality from dominating the judgment, a modal balance factor is introduced to control the modal contribution bias rate within ±12% after each round of fusion. The modal weighting coefficient is automatically adjusted through a feedback learning mechanism. 6.The AI-based adaptive scene intelligent interaction system according to claim 1, wherein, In the business scenario matching operation, the business scenario vector space is composed of 78-dimensional vectors. Each scenario node is generated through prior data training, Its feature vector contains four core features: business type identifier, scenario risk level, user historical satisfaction average, and context behavior similarity. The matching mechanism uses real-time cosine similarity calculation. If the similarity between the current user intent vector and multiple scenario nodes is higher than 0.85, the context judgment module is automatically started. Based on the business frequency priority at the current time point, user identity label, and emotional state score weighted decision, the optimal scenario node is selected. 7.The AI-based adaptive scene intelligent interaction system according to claim 1, wherein, In the scenario matching degree calculation, a high-level scoring mechanism that integrates user state perception and scenario structure features is introduced to improve the accuracy of the matching result and the real-time performance of the business response. The specific steps are as follows: Calculate the cosine similarity between the user's current intent vector and each business scenario node vector predicted by training to generate the initial semantic similarity score. The formula is as follows: wherein, is a similarity score between the th business scenario and the current intent of the user, is a user intent representation vector, is a th business scenario node vector, are Euclidean norms of vectors and respectively. Introduce the user's current emotional value and the urgency factor corresponding to the business scenario. Both participate in building the scenario response priority. Adjust the factor to control the influence weight on the final score. The calculation expression is as follows: wherein, is a scene response priority, is an emotional weight coefficient, is a user current emotional value, is a scene urgency weight coefficient, is a risk or urgency score of the nth scene node; The time-sensitive factor is added to depict the real-time urgency of the scene response, and the final scene comprehensive score is calculated, as shown in the following formula: In the formula, It is the final overall score for the scenario. It is the time sensitivity weighting coefficient. It is the natural base. Is the current time relative to the first Input variables for functions related to the urgency of each business scenario; Select with the highest The scene node is used as the final interaction target, and the scene executor is immediately dispatched to trigger the corresponding business process. At the same time, the data of each factor of this score and the user response feedback are synchronously sent back to the learning engine for subsequent strategy optimization.

Images