A hierarchical recognition method for intelligent customer service intentions in multimodal interaction

Abstract

This invention discloses a hierarchical recognition method for intelligent customer service intentions for multimodal interaction, including: acquiring and preprocessing three modal data types: text, speech, and image; constructing a high-dimensional topological structure of the modal data using a topological data analysis method and performing feature fusion; constructing a causal relationship diagram of modal features using a structural equation model to screen for features with significant causal contributions; employing a cross-modal self-supervised consistency regularization method to train an intention recognition model with minimal supervised data; and achieving multi-level hierarchical classification output of intentions based on a spectral clustering method. This invention significantly improves the accuracy and generalization performance of intention recognition in intelligent customer service systems, making it suitable for complex multimodal interaction scenarios.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and natural language processing technology, and in particular to a hierarchical recognition method for intelligent customer service intentions for multimodal interaction. Background Art

[0002] In recent years, with the rapid development of artificial intelligence (AI), intelligent customer service systems have been widely adopted in a variety of service industries, including communications, e-commerce, finance, transportation, and healthcare. By leveraging AI and natural language processing (NLP) technologies, these systems can identify and understand user intent, enabling automated responses and services, significantly improving service efficiency and effectively reducing labor costs.

[0003] Early intelligent customer service systems primarily relied on rule-based matching methods, which rely on manually crafted rules to identify user intent, such as keyword matching and sentence template recognition. While effective in specific scenarios, these methods were gradually replaced by more generalizable data-driven approaches due to high rule maintenance costs, limited generalization capabilities, and an inability to effectively handle undefined intents.

[0004] Currently, intent recognition technologies widely used in intelligent customer service are primarily based on machine learning and deep learning models. Traditional machine learning methods, such as support vector machines (SVMs) and naive Bayes (NBs), typically require manual feature design and have limited ability to recognize complex intents and context-sensitive questions. In recent years, deep learning methods, represented by deep neural networks, such as convolutional neural networks (CNNs), long short-term memory networks (LSTMs), and attention models, have been widely used in intent recognition tasks and have achieved significant improvements. These methods automatically extract features, effectively capturing the semantic information in user input and significantly improving the accuracy of intent recognition.

[0005] However, with the diversification of the ways in which users interact with intelligent customer service, the interaction scenarios have gradually expanded from single text or voice input to multimodal forms such as text, voice, images, and even videos. Multimodal interaction methods are more in line with real-world scenarios and provide richer semantic information. But at the same time, traditional intent recognition methods usually only perform feature extraction and classification on data of a single modality, lack an effective fusion mechanism for multimodal data, and find it difficult to fully utilize the complementary information between modalities. For example, existing multimodal fusion methods such as simple feature splicing, weighted summation, and average pooling can play a certain role in certain situations, but their fusion mechanisms are relatively simple, making it difficult to deeply explore the high-order intrinsic correlations between modalities, and easily causing redundancy of modal information or loss of effective information.

[0006] Furthermore, existing intelligent customer service intent recognition systems often lack explicit modeling of causal relationships between modal features, making it difficult to accurately assess the contribution of each modal information to the user's true intent. This results in a significant decrease in intent recognition accuracy and poor robustness in complex scenarios or with incomplete modal information. Furthermore, the training of most deep learning models requires the support of large amounts of labeled data. However, in practical applications, especially in emerging business scenarios, labeled data is often scarce or extremely costly. The lack of effective self-supervision or weak supervision mechanisms results in insufficient generalization performance, making it difficult to adapt to new service scenarios or low-sample requirements.

[0007] To address these issues, recent research has proposed methods based on self-supervised learning and transfer learning, enabling rapid model migration and generalization using small amounts of labeled data. While these methods have improved model training costs to a certain extent, they lack efficient consistency constraints between cross-modal features, and the design of multimodal data augmentation strategies and consistency loss functions remains imperfect, resulting in certain limitations in improving performance in multimodal environments.

[0008] Although existing technologies have achieved certain development results, they still have the following obvious shortcomings in actual complex multimodal interaction scenarios: the multimodal information fusion method is relatively simple and it is difficult to effectively mine high-order semantic associations between modalities; it fails to effectively model and utilize the causal relationship between modal features; the model generalization ability is poor, especially under the condition of a small amount of labeled data, it is difficult to maintain high intent recognition accuracy and robustness.

[0009] Therefore, how to provide an intelligent customer service intention hierarchical recognition method for multimodal interaction is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0010] One purpose of the present invention is to propose a hierarchical recognition method for intelligent customer service intentions for multimodal interactions. The present invention adopts a combination of topological data analysis (TDA), causal inference, self-supervised contrastive learning and spectral clustering methods, and effectively solves the problems of insufficient accuracy and poor generalization ability of user intention recognition in complex multimodal interaction scenarios in intelligent customer service systems through the following technical measures: First, three modal data of user interaction, namely text, voice and image, are collected, and preprocessing operations such as denoising, normalization and feature standardization are performed on each modal data to obtain clear and standardized multimodal data features; secondly, based on the persistent homology algorithm in topological data analysis, simplicial complexes are constructed for the feature spaces of each modal data after the above processing, and chain groups and boundary operators are calculated to obtain Betti numbers. The topological feature vectors composed of Betti numbers are used to realize the effective fusion of high-order semantic association features between different modalities; next, the structural equation model (SEM) method is used to construct an explicit causal relationship graph between the fused multimodal features, and the maximum likelihood estimation (MLE) is used to estimate the causal relationship between the fused multimodal features. The causal effect coefficient of each modal feature is calculated by using the Likelihood Estimation (MLE), which further clarifies the specific causal contribution of each modal feature to user intent. The modal features with low causal contribution are then eliminated to obtain the optimized multimodal fusion feature vector. Then, based on the optimized fusion feature vector, a cross-modal self-supervised consistency regularization method is adopted. Through three specific data augmentation strategies, namely random masking, random noise superposition, and random projection, a cross-modal self-supervised contrastive learning loss function is constructed to train the model. This enables the intent recognition model to be effectively trained with no or very little supervised data, thereby improving the generalization ability of the model. Finally, using the above-trained intelligent customer service intent recognition model, the Laplace eigenmap algorithm in spectral clustering is used to reduce the dimensionality of the feature vector and map it to a low-dimensional space to calculate the Euclidean distance. Clustering is then performed using the distance indicator to achieve accurate multi-level hierarchical recognition and classification output of user intent.The proposed method for hierarchical identification of intelligent customer service intentions for multimodal interaction has the following clear technical advantages: It effectively mines and utilizes high-order correlation information between modalities through a topological data analysis method, significantly enhancing the depth and accuracy of multimodal information fusion; it explicitly evaluates the causal contribution of each modal data feature to user intention through a structural equation model, avoiding the negative impact of redundant or interfering features on model accuracy; it effectively reduces the model's dependence on a large amount of labeled data through a cross-modal self-supervised consistency regularization method and a specific data augmentation strategy, significantly improving the model's generalization performance and robustness in environments with less labeled data; and it achieves precise hierarchical classification output of intent through a spectral clustering method, further enhancing the refinement and practical applicability of the intent recognition results of the intelligent customer service system. This invention effectively solves the problem of insufficient accuracy and generalization ability of intent recognition in complex multimodal interaction scenarios in the prior art intelligent customer service systems, significantly improving the application reliability and user service experience of the intelligent customer service system, making it applicable to various actual multimodal service scenarios and having good practical value.

[0011] According to an embodiment of the present invention, a method for hierarchical identification of intelligent customer service intentions for multimodal interaction is characterized by comprising the following steps:

[0012] S1. Obtain user interaction data, where the data includes three modalities: text, voice, and image, and perform preprocessing operations of denoising, normalization, and feature standardization on each modality data;

[0013] S2. For the modal data preprocessed in step S1, a persistent homology algorithm in topological data analysis is used to construct a high-dimensional topological structure between the modal data. The high-order correlation between the modalities is clearly quantified by the Betti number to form a topological feature representation. The topological features of each modality are then fused to obtain a preliminary fused topological feature vector.

[0014] S3. Based on the topological feature vector obtained in step S2, a causal inference method based on a structural equation model is used to construct an explicit causal relationship diagram between multimodal data features, quantitatively calculate the causal contribution weight of each modal feature to user intent recognition, eliminate features with no causal effect or negative causal effect, and only retain modal features with significant causal contribution for secondary fusion to form an optimized multimodal fusion feature vector.

[0015] S4. Using the optimized feature vector obtained in step S3 as input, a cross-modal self-supervised consistency regularization method based on contrastive learning is used to generate training sample pairs through data augmentation methods such as random masking, random noise superposition, and random projection. A cross-modal self-supervised contrastive learning loss function is constructed to optimize model parameters, thereby achieving effective training of the intelligent customer service intent recognition model with no or minimal supervised data.

[0016] S5. Using the intent recognition model trained in step S4, an intent distance measurement algorithm based on spectral clustering is used to achieve accurate multi-level hierarchical recognition and classification output of user intent in the multimodal fusion feature space.

[0017] Optionally, the S1 includes:

[0018] S1-1. Collect the original interaction data of the user in three modes: text, voice and image, which are represented as 、 and ;

[0019] S1-2. Text modal data Perform text denoising to remove stop words, special characters, and meaningless words to obtain clean text data ;

[0020] S1-3. Voice modality data Perform audio denoising and convert it to the spectrum domain using the short-time Fourier transform method to obtain the spectrum domain signal ,in represents the time frame, Represents the frequency, and then uses spectral subtraction to suppress background noise to obtain denoised speech data ;

[0021] S1-4. Image modality data The Gaussian filtering method is used to perform image denoising. The filtering operation expression is:

[0022] ;

[0023] Where, is a two-dimensional Gaussian function, represents the pixel coordinates of the image, is the Gaussian kernel standard deviation, and the denoised image data is obtained ;

[0024] S1-5, respectively normalize the denoised data obtained in steps S1-2 to S1-4, and obtain normalized data represented as 、 、 ;

[0025] S1-6. Perform feature standardization on the normalized data and calculate the standardized features using the z-score standardization method:

[0026] ;

[0027] Where, represents the standardized features, represents the normalized input features, represents the mean of the input features, Represents the standard deviation of the input features, and the standardized features are expressed as 、 、 .

[0028] Optionally, the S2 includes:

[0029] S2-1, constructing corresponding modal feature spaces based on the standardized feature data obtained in step S1;

[0030] S2-2. Use the continuous homology algorithm to construct the simplicial complex of each modal feature space, and define the chain group and boundary operator as:

[0031] ;

[0032] Where, represents the k-dimensional chain group, represents the boundary operator;

[0033] S2-3. Use chain groups and boundary operators to calculate the k-dimensional Betti number of each modal feature space:

[0034] ;

[0035] Where, is a closed chain group, For the boundary group;

[0036] S2-4, constructing topological eigenvectors using the Betti numbers of each modal eigenspace;

[0037] S2-5. Forming a preliminary fused topological feature vector based on topological feature vector splicing and fusion .

[0038] Optionally, the S3 includes:

[0039] S3-1, using the topological feature vector obtained in step S2 as input, constructing a causal relationship diagram between features using the SEM method;

[0040] S3-2. Based on the causal relationship diagram, establish the structural equation expression:

[0041] ;

[0042] Where, For nodes The corresponding modal features are Representation node The parent node set of For nodes To Node The causal effect coefficient, is subject to a mean of 0 and a variance of Normally distributed residual term;

[0043] S3-3. Use the maximum likelihood estimation (MLE) method to calculate the causal effect coefficient;

[0044] S3-4. Calculate the total causal contribution of modal features:

[0045] ;

[0046] Where, Representation node The collection of child nodes;

[0047] S3-5. Setting the threshold , eliminating those below features to form an optimized fusion feature vector .

[0048] Optionally, the S4 includes:

[0049] S4-1, the optimized multimodal fusion feature vector obtained in step S3 As the initial input, random masking, random noise superposition and random projection data enhancement are used to generate training sample pairs for cross-modal self-supervised contrastive learning. ,in and Respectively for the samples, feature vectors of two different views generated by random data augmentation;

[0050] S4-2. Define the cross-modal self-supervised contrastive learning loss function , the specific formula is:

[0051] ;

[0052] Where:

[0053] Indicates the number of sample pairs in each training batch;

[0054] Represents a vector 、 The cosine similarity between

[0055] The temperature hyperparameter is used to control the difficulty of training, and its value range is 0.05 to 0.2;

[0056] is the characteristic function, when The value is 1 when it is, otherwise the value is 0;

[0057] S4-3. Use the cross-modal self-supervised contrastive learning loss function defined in step S4-2 Optimize model parameters and obtain the final intelligent customer service intent recognition model through stochastic gradient descent training.

[0058] Optionally, the S4 includes:

[0059] S5-1. Use the intelligent customer service intention recognition model trained in step S4 to obtain the input feature vector The low-dimensional embedding representation vector , which is implemented using the Laplace eigenmap algorithm in spectral clustering. The specific calculation process is as follows:

[0060] First, construct the similarity matrix between feature vectors , the matrix elements Defined as:

[0061] ;

[0062] Where, is the width parameter of the Gaussian kernel function;

[0063] Next, calculate the matrix The degree matrix , the diagonal elements are defined as:

[0064] ;

[0065] Then calculate the normalized graph Laplacian matrix :

[0066] ;

[0067] Finally, solve the eigenvalue decomposition problem:

[0068] ;

[0069] Where, represents the feature vector, Indicates the eigenvalue, take the corresponding minimum The eigenvectors corresponding to the non-zero eigenvalues ​​constitute a low-dimensional embedding representation vector ;

[0070] S5-2, based on the low-dimensional embedding representation vector obtained in step S5-1 , calculate the distance metric of each user intention feature, specifically using the Euclidean distance formula:

[0071] ;

[0072] S5-3. Based on distance measurement indicators , perform spectral clustering on user intent and divide it into intent cluster subsets at different levels , to achieve accurate multi-level hierarchical recognition and classification output of user intentions.

[0073] Optionally, the voice modality data The preprocessing includes:

[0074] Firstly, the mel-frequency cepstral coefficient method is used to extract the spectral features of speech;

[0075] Then, based on the convolutional neural network, the time-frequency feature representation of the speech spectrum features is further extracted to obtain the standardized speech feature data. .

[0076] Optionally, the determination method in S3-5 is specifically:

[0077] Using cross-validation method, for different thresholds Calculate the recognition accuracy of the model separately , and finally select the optimal threshold that makes the model accuracy reach the maximum value ,Right now:

[0078] ;

[0079] Optionally, the intelligent customer service intention recognition system of the method includes:

[0080] The data acquisition and preprocessing module is used to acquire and preprocess the user's text, voice, and image interaction data to obtain standardized modal feature data;

[0081] Topological feature fusion module, used to perform topological data analysis on the preprocessed data and obtain the fused topological feature vector;

[0082] The causal inference module is used to perform causal relationship analysis between multimodal features, calculate the causal contribution of features and perform optimized fusion;

[0083] Intent model training module, used to train intelligent customer service intent recognition models based on cross-modal self-supervised contrastive learning methods;

[0084] The intention recognition and output module is used to achieve accurate multi-level hierarchical recognition and classification output of user intentions based on spectral clustering method.

[0085] The beneficial effects of the present invention are:

[0086] (1) The present invention adopts the persistent homology algorithm in the topological data analysis method to construct a high-dimensional topological structure between the modal data, and uses the Betti number as the topological feature to perform modal fusion, thereby achieving the effective capture and fusion of high-order semantic associations of multimodal data, effectively improving the accuracy and depth of multimodal data fusion, and enhancing the accuracy and stability of intent recognition of the intelligent customer service system in complex interaction scenarios.

[0087] (2) The present invention uses the structural equation model (SEM) to explicitly construct a causal relationship diagram between modal features, and quantitatively calculates the causal contribution weight of each modal feature based on the maximum likelihood estimation method, which can achieve effective feature selection and optimization fusion, significantly improve the pertinence and accuracy of modal feature fusion, and show better robustness and generalization ability in actual complex multimodal interaction environments.

[0088] (3) In terms of training the intent recognition model, the present invention adopts a cross-modal self-supervised consistency regularization method, specifically using data enhancement strategies such as random masking, random noise superposition, and random projection to construct a cross-modal self-supervised contrastive learning loss function. This effectively solves the problem of strong dependence of existing technologies on a large amount of labeled data, breaks through the bottleneck of poor model generalization ability in existing technologies, and achieves effective training and high generalization performance of models with a small amount or no labeled data, thereby effectively improving the technical level and practical engineering application capabilities of intelligent customer service systems in emerging or data-scarce service scenarios.

[0089] (4) The present invention uses the Laplace eigenmapping algorithm in the spectral clustering method to map the optimized fusion feature vector to a low-dimensional space and then cluster it, thereby achieving accurate multi-level hierarchical recognition and classification output of user intentions, significantly improving the granularity and refinement of user intention recognition, and demonstrating better service adaptability and user experience in actual multimodal interactive service application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0091] For example in the attached figure:

[0092] Figure 1 This is the overall flow chart of the intelligent customer service intention hierarchical recognition method for multimodal interaction proposed by the present invention;

[0093] Figure 2 This is a data preprocessing flow chart of a multimodal interaction-oriented intelligent customer service intention hierarchical recognition method proposed by the present invention;

[0094] Figure 3This is a schematic diagram of the topological data analysis and feature fusion process of the intelligent customer service intention hierarchical recognition method for multimodal interaction proposed by the present invention;

[0095] Figure 4 This is a schematic diagram of the causal inference and feature optimization process based on the structural equation model for the hierarchical identification method of intelligent customer service intentions for multimodal interaction proposed in the present invention;

[0096] Figure 5 This is a schematic diagram of the cross-modal self-supervised comparative learning process of the intelligent customer service intention hierarchical recognition method for multimodal interaction proposed in the present invention;

[0097] Figure 6 This is a schematic diagram of the spectral clustering-based intent hierarchical recognition method for intelligent customer service intention hierarchical recognition for multimodal interaction proposed in the present invention;

[0098] Figure 7 This is a schematic diagram of the structure of an intelligent customer service intention recognition system for a multimodal interaction-oriented intelligent customer service intention hierarchical recognition method proposed in the present invention. DETAILED DESCRIPTION

[0099] Regarding the multimodal data preprocessing described in step S1, it should be further clarified that:

[0100] During the preprocessing of text modality data, the stop word list must use the Chinese standard stop word list, which contains a total of 120 commonly used stop words. In the preprocessing of speech modality data, the short-time Fourier transform (STFT) window function uses a Hamming window with a window length of 25 ms and a frame shift step of 10 ms. The background noise in the spectral subtraction method is estimated by the average spectrum of the previous five frames of the signal. The two-dimensional Gaussian filter formula used in the preprocessing of image modality data is:

[0101] ;

[0102] The filter kernel size is fixed at 5×5 pixels, and the standard deviation =1.0, zero padding is used for the filter edge, and the pixel value normalization range is limited to the interval [0,1]. When constructing the topological structure for the continuous homology algorithm in step S2, the distance threshold used for topological feature vector calculation is fixed to 0.5, and the Euclidean distance is used for feature distance calculation. The specific calculation formula is:

[0103] ;

[0104] The vector dimension n is determined by the specific modal characteristics; the calculation of the Betti number is limited to 0-dimensional and 1-dimensional characteristics, and the boundary operator is calculated using modulo-2 addition. When constructing the structural equation model in step S3, the initial structural equation graph is assumed to be a fully connected directed acyclic graph, and the structural equation between each node is expressed as:

[0105] ;

[0106] Where pa(i) represents the set of parent nodes of node i. The causal contribution weight threshold is set to 0.05 after cross-validation, and features below this threshold are deleted. The random data augmentation operation of cross-modal self-supervised consistency regularization in step S4 is specifically set as follows: the random masking ratio is fixed to 20%, the standard deviation of random Gaussian noise is fixed to 0.01, and the dimension reduced by the random projection operation is fixed to 10% of the original dimension; the cross-modal self-supervised contrastive learning loss function is defined as:

[0107] ;

[0108] The temperature hyperparameter The similarity matrix in the spectral clustering method in step S5 is constructed using a Gaussian kernel function, which is specifically defined as:

[0109] ;

[0110] The Gaussian kernel width parameter The number of spectral clustering categories is fixed to 5, the standard k-means algorithm is used to implement clustering, the number of iterations is set to 200, and the convergence criterion threshold is fixed to .

[0111] This method effectively captures and fuses high-order correlations and explicit causal relationships between multimodal data through topological data analysis, structural equation modeling, and cross-modal self-supervised consistency regularization, significantly improving the accuracy of intent recognition and the generalization performance of the model. Furthermore, the use of spectral clustering for hierarchical output of intent enhances the refinement and response efficiency of user intent classification, effectively overcoming the shortcomings of existing technologies in terms of insufficient fusion mechanisms and limited generalization performance.

[0112] Supplementary explanation of the data preprocessing technical details described in step S1:

[0113] Text modal raw interaction data Specifically collected from Chinese natural language text input by users; original interaction data in voice mode The acquisition adopts PCM audio equipment with a sampling rate of 16kHz and a quantization bit of 16 bits; the original interactive data of the image modality The image is collected using an RGB image sensor with a fixed resolution of 1920×1080 pixels. The stop word list used in the text data denoising process in S1-2 contains a total of 120 Chinese stop words, and the special character removal set includes punctuation marks and symbols totaling 20 characters. The short-time Fourier transform (STFT) in the speech data denoising in step S1-3 uses a Hamming window with a window length of 25ms and a frame shift step of 10ms. The background noise spectrum used in the spectral subtraction is accurately calculated by the mean of the spectra of the first five frames of the audio signal. The two-dimensional Gaussian filter in the image denoising in S1-4 uses a filter kernel with a fixed size of 5×5 pixels and a standard deviation of Set to 1.0, the image edge processing adopts the zero filling method, and the pixel value normalization processing of the filtered image data is precisely limited to the [0,1] interval. The z-score normalization formula used in step S1-6 is specifically expressed as:

[0114] ;

[0115] in is the mean of all feature data, is the standard deviation of the characteristic data. The characteristic mean and standard deviation are calculated accurately to 4 decimal places.

[0116] The present invention implements strict and meticulous data denoising, normalization and feature standardization preprocessing on text, voice and image modal data, so that different modal data can be input into the subsequent analysis process with consistent scale and high quality, effectively improving the reliability and accuracy of data fusion, significantly improving the stability and robustness of intent recognition in multimodal intelligent customer service systems, and overcoming the problem of insufficient fusion accuracy caused by inconsistent data quality in existing technologies.

[0117] Regarding the data analysis details of step S2, the dimensions of each modal feature space should be clearly defined in the specific implementation:

[0118] The dimension of the text modality feature space is 300, the dimension of the speech modality feature space is 13, and the dimension of the image modality feature space is 1024. The simplicial complex construction method used by the persistent homology algorithm is the Vietoris-Rips method, in which the distance between data points is strictly calculated using the Euclidean distance, which is calculated as follows:

[0119] ;

[0120] in 、 For any two data points in the modal feature space, dimension is the dimension of the above modal feature space. The threshold distance for constructing simplicial complexes in the Vietoris-Rips method is set to 0.5. and boundary operators The calculation of is strictly implemented according to the modulo-2 addition rule. The Betti number calculation is strictly limited to 0-dimensional and 1-dimensional Betti numbers, where the 0-dimensional Betti number represents the number of connected components and the 1-dimensional Betti number represents the number of ring structures. The Betti number eigenvector obtained for each mode is specifically expressed as:

[0121] ;

[0122] That is, it is formed by concatenating the 0-dimensional and 1-dimensional Betti numbers of each mode in the above order.

[0123] The present invention uses a persistent homology algorithm to quantitatively analyze the topological structure of the multimodal data feature space to accurately construct high-order topological feature vectors, effectively revealing the deep correlation between each modal data at the topological level, significantly improving the accuracy of subsequent feature fusion and intelligent customer service intention recognition, and overcoming the problems of feature information redundancy and missing important related information caused by existing simple splicing or weighted fusion methods.

[0124] Regarding the construction process of the structural equation model (SEM) causal relationship diagram in step S3, the following technical details should be clearly supplemented:

[0125] When the initial causal relationship graph is constructed, all topological feature nodes form a fully connected directed acyclic graph, and the number of nodes is consistent with the dimension of the topological feature vector obtained in step S2. The calculation of is solved using the maximum likelihood estimation (MLE) method. Specifically, the log-likelihood function that maximizes the observed data is expressed as:

[0126] ;

[0127] in 、 Respectively represent Nodes in the observation sample 、 The eigenvalues ​​of Represents the total number of observation samples and the variance of the residual term of each node In the process of model solution, the initial setting is 1.0, and the specific value of the residual variance of each node is calculated by MLE iteration. When calculating the total causal contribution of modal characteristics, the node subnode set The causal contribution threshold is determined according to the final causal relationship graph structure. The causal contribution is precisely determined to be 0.05 through cross-validation. Features with a value lower than 0.05 are deleted from the fused feature vector, and the remaining features form the optimized fused feature vector. .

[0128] The present invention uses a structural equation model to finely construct a clear causal relationship between various topological features, and strictly adopts the maximum likelihood estimation method to calculate the causal effect coefficient between each node, quantitatively evaluates the actual contribution of each modal feature to the user intention recognition task, accurately screens effective features, effectively avoids the interference of redundant and negatively impacting features, significantly enhances the generalization ability and robustness of the model, and effectively overcomes the problem of poor intent recognition accuracy caused by inaccurate causal relationship analysis in the existing technology.

[0129] Regarding the cross-modal self-supervised contrastive learning training details described in step S4, the specific implementation process is as follows:

[0130] The random masking data enhancement method specifically uses random masking of 20% of the elements in each sample feature vector, and the masking position is uniformly randomly selected; the random noise superposition method is to superimpose independent and identically distributed Gaussian noise with a mean of 0 and a fixed standard deviation of 0.01 on each element of the feature vector; the random projection data enhancement method specifically uses a random Gaussian projection matrix to reduce the original feature dimension to 90% of the original dimension, where the elements of the random projection matrix are distributed from the standard normal distribution. The cross-modal self-supervised contrastive learning loss function is specifically defined as:

[0131] ;

[0132] The number of samples in a batch The specific fixed value is 64, the temperature hyperparameter The fixed value is 0.1, and the model parameter optimization algorithm explicitly adopts the stochastic gradient descent method, in which the initial learning rate is fixed to 0.001, the number of iterations of the optimization process is specifically set to 1000 times, and the gradient convergence threshold is fixed to 10^{-6}.

[0133] This invention combines a cross-modal self-supervised consistency regularization method with a specific data augmentation strategy to effectively train an intent recognition model with multimodal fusion features under conditions of unsupervised or minimally supervised data, significantly improving the model's generalization performance and recognition accuracy. It breaks through the problem in the existing technology that the model's generalization ability is limited by the reliance on a large amount of labeled data, and has stronger practical application capabilities.

[0134] The specific implementation details of the spectral clustering method in step S5 are further clarified as follows:

[0135] Similarity matrix in spectral clustering The construction uses Gaussian kernel function, kernel width parameter The specific value of is fixed to 1.0, and the dimension of the input optimized fusion feature vector is 128. is a diagonal matrix with diagonal elements Specifically, by The sum of each row of elements is obtained, that is:

[0136] ;

[0137] Normalized graph Laplacian matrix The specific calculation is:

[0138] ;

[0139] Solve the eigenvalue decomposition problem:

[0140] ;

[0141] Specifically, the eigenvalue decomposition method is used. After sorting from small to large, take the eigenvectors corresponding to the first 5 smallest non-zero eigenvalues ​​as the low-dimensional embedding representation vector The distance index of the intent feature vector explicitly uses the Euclidean distance calculation formula:

[0142] ;

[0143] The specific clustering operation of spectral clustering adopts the standard k-means algorithm, in which the number of cluster categories is fixed to 5, the maximum number of iterations is explicitly set to 200 times, and the convergence threshold is fixed to .

[0144] The present invention effectively realizes multi-level hierarchical and accurate recognition of user intentions through the spectral clustering method. The similarity matrix constructed by the Gaussian kernel function and the Laplace eigenmap are used to map high-dimensional fusion features to low-dimensional space to improve the stability and accuracy of clustering. The refinement of the intention recognition results of the intelligent customer service system is significantly enhanced, the practical applicability of the system and the user experience are significantly improved, and the shortcomings of existing methods that cannot achieve accurate and efficient hierarchical recognition are effectively overcome.

[0145] The specific implementation details of speech modality data preprocessing are further clarified as follows:

[0146] In the specific calculation process of Mel-frequency cepstral coefficients (MFCC), 26 Mel-scale filters are used. The time-frequency domain representation of the audio signal is converted by short-time Fourier transform (STFT) and then passes through a Mel filter bank. The logarithmic energy value is then taken, and the discrete cosine transform (DCT) is used to extract the first 13 cepstral coefficients as speech spectrum features. The subsequent convolutional neural network (CNN) structure is specifically a network containing two one-dimensional convolutional layers. The first convolutional layer contains 32 convolution kernels, each with a size of 3 and a step size of 1; the second convolutional layer contains 64 convolution kernels, each with a size of 5 and a step size of 1. After each convolution layer, the ReLU activation function is used, and a maximum pooling operation with a kernel size of 2 is performed. Finally, a fully connected layer is used to map the convolution pooled features to a standardized speech feature vector with a dimension of 128 to form a data. .

[0147] The present invention uses the method of combining Mel-frequency cepstral coefficients with convolutional neural networks to finely extract the spectrum and time-frequency feature representation of speech modal data, fully capturing the key features of the frequency and time dimensions of the speech signal, effectively improving the accuracy and stability of the speech feature representation, and significantly enhancing the support capability of speech modal data for subsequent multimodal fusion and intent recognition tasks, overcoming the problems of the existing speech feature extraction methods being single and having limited generalization capabilities.

[0148] Feature selection threshold for step S3-5 The specific implementation details of the determination method are further clarified as follows:

[0149] The threshold value was determined using the ten-fold cross validation method. The selection process, in which the threshold value range is strictly set from 0.01 to 0.1, the interval step is 0.01, and the recognition accuracy of the model is calculated on the cross-validation data set for each threshold. The data for each cross-validation is precisely divided into 90% for the training set and 10% for the validation set. Feature selection threshold Finally, the specific value corresponding to the maximum value of the ten-fold cross-validation average accuracy is clearly determined:

[0150] ;

[0151] The present invention determines the feature selection threshold in the structural equation model through a precise cross-validation method, and can objectively and accurately evaluate the impact of model feature selection on recognition performance under different thresholds, significantly optimizing the scientificity and effectiveness of feature selection, avoiding errors and performance fluctuations caused by artificially set thresholds, effectively improving the generalization ability of the model and the accuracy of intent recognition, and overcoming the problem of the lack of precise standards in the feature selection methods of the existing technology.

[0152] For the intelligent customer service intention recognition system, the specific implementation details of each module are further clarified as follows:

[0153] In the data acquisition and preprocessing module, text data was denoised using a standard Chinese stop word list of 120 stop words. Speech data was collected using a PCM audio device with a sampling rate of 16kHz and 16-bit quantization. Preprocessing used a short-time Fourier transform (STFT) with a window length of 25ms and a frame shift step of 10ms. Spectral subtraction used the mean of the spectra of the first five audio frames as the background noise estimate. Image data was collected using a 1920×1080 pixel RGB image sensor and preprocessed using a 5×5 pixel two-dimensional Gaussian filter kernel with a standard deviation of 1.0 and normalized to the interval [0,1]. All data features were standardized using the z-score method. The topological feature fusion module constructed a simplicial complex strictly based on the Vietoris-Rips method, with a threshold distance of 0.5. Feature vector calculation was limited to 0- and 1-dimensional Betti numbers. The causal inference module uses a structural equation model (SEM), initially set as a fully connected directed acyclic graph (DAG). Causal effect coefficients are calculated strictly using the maximum likelihood estimation (MLE) method, and the causal contribution threshold is strictly determined by cross-validation to be 0.05. The intent model training module uses cross-modal self-supervised contrastive learning, with a random masking ratio of 20%, a random Gaussian noise standard deviation of 0.01, and a random projection dimension reduced to 10% of the original dimension. The temperature parameter in the contrastive learning loss function is fixed at 0.1, and the number of training batches is 64. The intent recognition and output module uses spectral clustering to construct a similarity matrix using a Gaussian kernel function with a width parameter of 1.0. The number of cluster categories is fixed at 5, and the k-means algorithm is used for 200 iterations. The convergence threshold is set to 10^{-4}.

[0154] The intelligent customer service intention recognition system proposed in the present invention achieves efficient and accurate multimodal data fusion and intention recognition through efficient data preprocessing, topological data analysis, clear causal relationship modeling and cross-modal self-supervised learning strategies, significantly improving the system's generalization ability and intent recognition precision in complex multimodal interaction environments, overcoming the problems of simple fusion methods, blind feature selection and lack of robustness in existing technologies, and has significant technical advantages and practical application value.

[0155] Example 1:

[0156] A hospital's online intelligent consultation platform has introduced the present invention's intelligent customer service intention hierarchical recognition method for multimodal interaction to improve the accuracy of understanding patients' consultation intentions. In this scenario, patients often interact with the system using voice, text, and images at the same time. For example, patients may describe their symptoms by voice, while entering text in a dialog box to supplement their condition information, or upload relevant image materials such as photos of the affected area. Multimodal input provides the system with rich information, but also brings challenges in understanding the user's true intentions. Traditional single-modal intent recognition models are prone to misjudgment when faced with such complex inputs, and are unable to capture the conversation context in a timely manner, resulting in delayed answers or irrelevant answers. For example, a patient may vaguely state that "I have been feeling unwell recently" but do not specify the specific symptoms; or the patient may describe a rash but do not upload photos. In these cases, ordinary intelligent consultation systems may find it difficult to understand the questions patients want to consult in a timely and accurate manner.

[0157] This embodiment optimizes the aforementioned challenges through the method of the present invention. In terms of system architecture, the intelligent customer service first uses speech recognition technology to transcribe the patient's spoken description into text and extract emotions and key information from the speech (e.g., intonation can reflect pain intensity). Simultaneously, the system applies image recognition models to analyze images uploaded by the patient (e.g., photos of wounds or rashes) and extracts feature information related to the medical intent (e.g., the location of the affected area, severity, or possible symptom characteristics). The platform then fuses the multimodal data into a unified semantic representation. Based on the hierarchical intent recognition model of the present invention, the system analyzes user intent at two levels: the first level identifies the high-level intent categories of the patient's inquiry (e.g., "symptom consultation," "appointment registration," "report interpretation," etc.). The second level then identifies specific intents based on specific semantic details (e.g., after identifying "symptom consultation," it then determines whether the inquiry is about "itchy skin" or "cough and fever," etc.). This hierarchical recognition process fully leverages the information contained in speech, text, and images, providing a more comprehensive understanding of patient needs than relying solely on text analysis. Notably, when patients provide images, the system can cross-correlate the image content with the text description, increasing confidence in the intent assessment. For example, if the voice text description mentions a rash, and the image analysis results also indicate a skin rash, the method of this invention will comprehensively identify the intent as "consulting about skin symptoms," significantly improving recognition accuracy.

[0158] The multimodal hierarchical intent recognition of this embodiment demonstrates robustness against ambiguous patient responses or incomplete information. For ambiguous inquiries, the system first uses first-level intent recognition to categorize the question into a broad category. If confidence is low, clarifying questions are then asked during the conversation to guide the patient in providing more information. For example, if a patient simply says, "I feel a little uncomfortable in my stomach," the system initially categorizes it as a "Symptom Consultation - Digestive" intent. However, due to the general nature of the description, the system then asks, "Are you experiencing stomach pain or diarrhea? How long have the symptoms lasted?" This multi-round interaction strategy is effectively integrated into the method of this invention as part of the intent recognition process. By gradually refining the user's intent, the system can address the patient's initial vague description and reduce misunderstandings. Furthermore, if information from a particular modality is missing, this method intelligently compensates: if the user does not provide an image, the system does not stop the recognition process, but instead relies more on the content and context of the spoken text to determine intent. Conversely, if the voice input is unclear due to ambient noise, the system prioritizes the user's subsequent text input. In this way, even if there is a certain modality missing or incomplete information, the intent hierarchical recognition model can still make up for it through information sources of other modalities, thereby maintaining a high accuracy of understanding user intent.

[0159] The method of the present invention also greatly improves the ability to understand intent in multi-round medical consultation dialogues. Previous online medical consultation systems often have the problem of poor context tracking in long conversations, resulting in the patient's true intention in multiple rounds of consultation not being accurately captured. In this embodiment, the intelligent customer service uses the intention hierarchical model to dynamically adjust its understanding of the user's intent in each round of response. For example, the patient first asks "Do I need to go to the hospital for this rash?" The system recognizes that its intention is "symptom consultation-medical advice." When the patient then asks "Which department do I need to go to?", the system can realize that the user's focus has shifted from "whether medical treatment is needed" to "guidance on medical departments." Due to the previous context, the system identifies the new question as the "medical guidance" intent in the second round, and links it back to the rash scenario discussed in the previous round, thereby giving a coherent and correct answer. Throughout the process, the model of the present invention achieves accurate understanding of multiple rounds of questions by maintaining the semantic context and hierarchical intent labels of the conversation, avoiding context loss or misjudgment.

[0160] After deploying the multimodal hierarchical intent recognition method of the present invention, the performance indicators of the hospital's intelligent medical consultation platform have significantly improved. Table 1, "Improvement Data for Medical Consultation Intent Recognition," provides a comparison of key indicators before and after deployment. As can be seen, intent recognition accuracy increased from 85% before deployment to 95% after deployment, demonstrating a significant improvement. This is due to the more comprehensive understanding provided by multimodal information fusion, as well as the effective reduction of errors caused by ambiguity in the hierarchical recognition strategy. Furthermore, the average response time was shortened from 5.2 seconds to 3.1 seconds, meaning the system can provide targeted responses more quickly. This is due in part to the improved accuracy of initial understanding, which reduces rounds of repeated clarifications, and in part to the optimized response efficiency of multimodal parallel processing. The misunderstanding rate dropped from 12% to 2.5%, indicating a significant reduction in misinterpretations of user intent. This reduction in misunderstandings directly improves the user consultation experience and reduces the frustration caused by irrelevant answers. The above data demonstrates that the application of the present method has brought significant benefits to medical consultation scenarios: it not only improves the intelligent customer service's ability to accurately grasp patient intent but also accelerates response times, demonstrating its high engineering value.

[0161] Table 1 below summarizes the specific numerical changes of various indicators in this embodiment:

[0162] Table 1 Improved data of medical consultation intention recognition

[0163]

[0164] The performance improvement brought about by the method of the present invention can be intuitively seen from Table 1. The significant improvement in the accuracy of intent recognition indicates that the system understands the patient's consultation intention more reliably, which is directly related to the reduction of understanding bias and misunderstanding by multimodal information fusion. The shortened response time reflects the improvement in system efficiency - patients receive useful replies faster, making the consultation process smoother. The significant reduction in the misunderstanding rate is particularly critical, which means that the proportion of incorrect answers or failure to understand the patient's meaning is significantly reduced, reducing the patient's need for repeated explanations and the potential dissatisfaction caused by this. Overall, this embodiment proves that the intention layered recognition method for multimodal interaction can effectively improve the accuracy and reliability of the dialogue system in the field of medical consultation, and provide patients with more accurate and efficient intelligent consultation services.

[0165] Example 2:

[0166] In this embodiment, the multimodal interaction intention layered recognition method of the present invention is deployed in the intelligent customer service system of a large e-commerce platform to optimize the understanding and response of user inquiries. User inquiries on e-commerce platforms usually have multimodal interaction characteristics: users may describe the problems they encounter through voice (for example, voice inquiries about product information or order status), enter text in the dialog box (such as providing order numbers or specific requests), and upload relevant pictures (such as invoices, photos of product defects, etc.) to assist in explaining the situation. Faced with such a wealth of input information, if each modality is processed independently, traditional customer service systems often find it difficult to grasp the core demands of users in a timely and accurate manner, and may even make misjudgments due to inconsistent information from multiple sources.

[0167] After introducing the method of the present invention, the system first recognizes and transcribes the user's voice content and analyzes the emotions and key points in the voice. For example, detecting that the user's tone is anxious may indicate an urgent problem. The text input part directly extracts key fields, such as order number, product name, etc. For images uploaded by users, the system uses image recognition and OCR technology to extract useful information: if the image is an invoice or order screenshot, transaction data such as order number and date are extracted; if it is a product photo, features such as product type or defective parts are identified. Subsequently, intelligent customer service integrates information from the three modalities of voice, text, and image to represent the user's current intention state in a unified semantic space. Based on the intention hierarchical model, the system also performs two-level intent recognition: the first level determines the main category of the user's help request, such as "product consultation", "logistics inquiry", "return and exchange" or "account / payment problem"; the second level further subdivides the specific intent after locking the major category. For example, when the first-level recognition result is the "return or exchange" category, the system will combine the context to determine whether the user's specific intention is to "apply for a return," "inquire about the return progress," or "understand the return policy," and adopt different response strategies accordingly. Compared with the previous practice of simply mapping a sentence to a fixed intention, this hierarchical recognition fully utilizes the complementary advantages of different modal information, making the judgment more accurate and more context-adaptive. In particular, when the user provides multiple pieces of evidence, this method can cross-verify the information: for example, if the user says "I want to return the product" and the text also provides the corresponding order number, the system will associate the intention in the voice with the order details provided by the text, accurately identify the user's intention as "initiate an order return," and locate the products and orders involved. The fusion of multimodal information here reduces the errors that may occur in a single modality and significantly improves recognition accuracy and reliability.

[0168] This embodiment highlights the system's ability to handle modal redundancy and conflict. When redundancy exists between a user's multiple input modalities, the system considers this as a favorable factor for information verification. For example, if a user both describes a problem verbally and repeats key information (such as an order number) in text, the system will compare the two inputs for consistency. If there are slight discrepancies between the voice recognition result and the text, the system can analyze and correct these discrepancies, thus avoiding misunderstandings caused by voice recognition errors. When modal information is inconsistent or even conflicting, the method of the present invention enables strategic fusion judgment, prioritizing the user's primary intent while not neglecting the meaning implied by secondary information. For example, a user may say, "I want to return this phone because it's defective," but the uploaded image is a purchase receipt. While the voice intent appears to indicate a return, the image represents a transaction receipt, and the two do not literally match. Traditional systems may be confused as to whether a user wants to return a product or inquire about an invoice. However, our approach's layered intent recognition first identifies the primary intent as "return request" based on voice and text. It then uses the invoice image as a supplementary tool to extract order and product information, assisting in the return process, rather than misclassifying it as a separate intent related to "invoice inquiry." This demonstrates that even if the information provided by the modalities superficially mismatches, the system can still correctly interpret the user's true needs through intent layering and multimodal fusion. If multiple independent requests are conveyed through different modalities, our approach can also identify the underlying complex intents and differentiate them hierarchically. For example, if a user simultaneously requests "inquiry about product performance" and "request an invoice," the system will first identify multiple intents and categorize them into "product inquiry" and "after-sales service," respectively. It then provides responses to each of these intents during the conversation, thus enabling multiple answers to a single question.

[0169] Through the above mechanism, e-commerce customer service systems demonstrate greater intelligence and flexibility during multi-turn interactions. Users' questions are often sequential and related, such as asking, "Does this product have a warranty?" and then, after receiving an answer, immediately asking, "How do I apply for a return?" The method of the present invention enables the system to track such shifts in intent: the first question is identified as "product inquiry - warranty policy," while the second is identified as "returns and exchanges - return process." Recognizing this as a continuous request from the same user within the same conversation, the system automatically switches the context from product information to the return issue, while retaining previous information related to the product to answer related details. This ability to track intent in multi-turn conversations stems from the dynamic context adjustment of the intent hierarchy model, ensuring that each question is accurately mapped to the corresponding intent category, without losing direction due to topic changes. Traditional customer service robots may require users to provide repeated information in this scenario or directly transfer the conversation to manual processing. However, applying the method of the present invention, the system can autonomously handle more complex multi-turn conversations, continuously understanding and responding to different user requests.

[0170] After deployment and testing on an e-commerce platform, the method presented in this paper significantly improved various performance indicators of the customer service system. Table 2, below, lists the performance improvement data for customer service intent recognition on the e-commerce platform, comparing key indicators before and after the improvements. As can be seen, the accuracy of hierarchical intent recognition increased from 88% to 96%, demonstrating that the system is more accurate in assessing user intent. This increased accuracy directly reduces irrelevant answers and reduces the need for manual intervention. User satisfaction scores also significantly improved (for example, from 80% to 94%), with users generally reporting that the new intelligent customer service system better understood their requests and provided more tailored responses. Furthermore, the average response time was reduced from 4.5 seconds to 2.7 seconds, significantly reducing the time users spent waiting for answers after asking questions. This faster response is due to the method's higher recognition efficiency and optimized process—the system is able to instantly extract key information from multimodal input and quickly retrieve answers, reducing intermediate processing steps. This faster response not only makes users feel the service is more efficient but also further improves their satisfaction. Finally, the user complaint rate dropped significantly from 7.2% before deployment to 1.5%. This significantly reduces user dissatisfaction and complaints due to misunderstandings or inappropriate responses, fully demonstrating the reliability and superiority of the proposed method in practical applications. Overall, the data in Table 2 clearly demonstrates that the multimodal interaction intent layering recognition method brings higher accuracy, faster response speeds, and better user evaluations to e-commerce customer service systems, demonstrating extremely high engineering and commercial value.

[0171] Table 2 Performance improvement data of customer service intention recognition on e-commerce platforms

[0172]

[0173] Table 2 clearly demonstrates the significant improvement in the effectiveness of the present method in e-commerce customer service scenarios. The improved accuracy of intent recognition means that intelligent customer service can understand customer requests almost flawlessly, making accurate judgments even when faced with complex information containing voice, text, and images. This ensures that customer questions are accurately answered and reduces back-and-forth communication caused by misunderstandings. The significant increase in user satisfaction reflects customer recognition and satisfaction with the new customer service system. This is because the system provides more accurate answers and efficient service, allowing customers to feel that communication is smooth and their problems are truly resolved. The reduction in average response time further enhances the user experience—faster responses make customers feel that the service is efficient and valued. The decline in complaint rates indicates that customer complaints previously caused by robot misunderstandings or ineffective responses have been virtually eliminated, demonstrating the reliability of the present method in real-world business environments. In summary, Example 2 demonstrates that the multimodal interactive intent layered recognition method can effectively integrate multiple sources of information, such as voice, text, and images, in the e-commerce customer service field to accurately understand customer intent and respond promptly, not only comprehensively improving service quality indicators but also enhancing user trust and satisfaction with the platform's customer service.

Claims

1. A hierarchical identification method for intelligent customer service intentions for multimodal interaction, characterized by: The steps include: S1. Obtain user interaction data, where the data includes three modalities: text, voice, and image, and perform preprocessing operations of denoising, normalization, and feature standardization on each modality data; S2. For the modal data preprocessed in step S1, a persistent homology algorithm in topological data analysis is used to construct a high-dimensional topological structure between the modal data. The high-order correlation between the modalities is clearly quantified by the Betti number to form a topological feature representation. The topological features of each modality are then fused to obtain a preliminary fused topological feature vector. S3. Based on the topological feature vector obtained in step S2, a causal inference method based on a structural equation model is used to construct an explicit causal relationship diagram between multimodal data features, quantitatively calculate the causal contribution weight of each modal feature to user intent recognition, eliminate features with no causal effect or negative causal effect, and only retain modal features with significant causal contribution for secondary fusion to form an optimized multimodal fusion feature vector. S4. Using the optimized feature vector obtained in step S3 as input, a cross-modal self-supervised consistency regularization method based on contrastive learning is used to generate training sample pairs through data augmentation methods such as random masking, random noise superposition, and random projection. A cross-modal self-supervised contrastive learning loss function is constructed to optimize model parameters, thereby achieving effective training of the intelligent customer service intent recognition model with no or minimal supervised data. S5. Using the intent recognition model trained in step S4, an intent distance measurement algorithm based on spectral clustering is used to achieve accurate multi-level hierarchical recognition and classification output of user intent in the multimodal fusion feature space; Step S1 specifically includes the following sub-steps: S1-1. Collect the original interaction data of the user in three modes: text, voice and image, which are represented as 、 and ; S1-2. Text modal data Perform text denoising to remove stop words, special characters, and meaningless words to obtain clean text data ; S1-3. Voice modality data Perform audio denoising and convert it to the spectrum domain using the short-time Fourier transform method to obtain the spectrum domain signal ,in represents the time frame, Represents the frequency, and then uses spectral subtraction to suppress background noise to obtain denoised speech data ; S1-4. Image modality data The Gaussian filtering method is used to perform image denoising. The filtering operation expression is: ； Where, is a two-dimensional Gaussian function, represents the pixel coordinates of the image, is the Gaussian kernel standard deviation, and the denoised image data is obtained ; S1-5, respectively normalize the denoised data obtained in steps S1-2 to S1-4, and obtain normalized data represented as 、 、 ; S1-6. Perform feature standardization on the normalized data and calculate the standardized features using the z-score standardization method: ； Where, represents the standardized features, represents the normalized input features, represents the mean of the input features, Represents the standard deviation of the input features, and the standardized features are expressed as 、 、 ; Step S2 specifically includes the following sub-steps: S2-1, constructing corresponding modal feature spaces based on the standardized feature data obtained in step S1; S2-2. Use the continuous homology algorithm to construct the simplicial complex of each modal feature space, and define the chain group and boundary operator as: ； Where, represents the k-dimensional chain group, represents the boundary operator; S2-3. Use chain groups and boundary operators to calculate the k-dimensional Betti number of each modal feature space: ； Where, is a closed chain group, For the boundary group; S2-4, constructing topological eigenvectors using the Betti numbers of each modal eigenspace; S2-5. Forming a preliminary fused topological feature vector based on topological feature vector splicing and fusion .

2. The method according to claim 1, characterized in that Step S3 specifically includes the following sub-steps: S3-1, using the topological feature vector obtained in step S2 as input, constructing a causal relationship diagram between features using the SEM method; S3-2. Based on the causal relationship diagram, establish the structural equation expression: ； Where, For nodes The corresponding modal features are Representation node The parent node set of For nodes To Node The causal effect coefficient, is subject to a mean of 0 and a variance of Normally distributed residual term; S3-3. Use the maximum likelihood estimation (MLE) method to calculate the causal effect coefficient; S3-4. Calculate the total causal contribution of modal features: ； Where, Representation node The collection of child nodes; S3-5. Setting the threshold , eliminating those below features to form an optimized fusion feature vector .

3. The method according to claim 1, characterized in that Step S4 specifically includes the following sub-steps: S4-1, the optimized multimodal fusion feature vector obtained in step S3 As the initial input, random masking, random noise superposition and random projection data enhancement are used to generate training sample pairs for cross-modal self-supervised contrastive learning. ,in and Respectively for the samples, feature vectors of two different views generated by random data augmentation; S4-2. Define the cross-modal self-supervised contrastive learning loss function , the specific formula is: ； Where: Indicates the number of sample pairs in each training batch; Represents a vector 、 The cosine similarity between The temperature hyperparameter is used to control the difficulty of training, and its value range is 0.05 to 0.2; is the characteristic function, when The value is 1 when it is, otherwise the value is 0; S4-3. Use the cross-modal self-supervised contrastive learning loss function defined in step S4-2 Optimize model parameters and obtain the final intelligent customer service intent recognition model through stochastic gradient descent training.

4. The method according to claim 1, wherein Step S5 specifically includes the following sub-steps: S5-1. Use the intelligent customer service intention recognition model trained in step S4 to obtain the input feature vector The low-dimensional embedding representation vector , which is implemented using the Laplace eigenmap algorithm in spectral clustering. The specific calculation process is as follows: First, construct the similarity matrix between feature vectors , the matrix elements Defined as: ； Where, is the width parameter of the Gaussian kernel function; Next, calculate the matrix The degree matrix , the diagonal elements are defined as: ； Then calculate the normalized graph Laplacian matrix : ； Finally, solve the eigenvalue decomposition problem: ； Where, represents the feature vector, Indicates the eigenvalue, take the corresponding minimum The eigenvectors corresponding to the non-zero eigenvalues ​​constitute a low-dimensional embedding representation vector ; S5-2, based on the low-dimensional embedding representation vector obtained in step S5-1 , calculate the distance metric of each user intention feature, specifically using the Euclidean distance formula: ； S5-3. Based on distance measurement indicators , perform spectral clustering on user intent and divide it into intent cluster subsets at different levels , to achieve accurate multi-level hierarchical recognition and classification output of user intentions.

5. The method according to claim 1, characterized in that The voice modality data The preprocessing specifically includes: Firstly, the mel-frequency cepstral coefficient method is used to extract the spectral features of speech; Then, based on the convolutional neural network, the time-frequency feature representation of the speech spectrum features is further extracted to obtain the standardized speech feature data. .

6. The method according to claim 2, characterized in that Feature selection threshold in step S3-5 The specific method for determining is: Using cross-validation method, for different thresholds Calculate the recognition accuracy of the model separately , and finally select the optimal threshold that makes the model accuracy reach the maximum value ,Right now: .

7. An intelligent customer service intention recognition system implementing the method of claim 1, characterized in that: include: The data acquisition and preprocessing module is used to acquire and preprocess the user's text, voice, and image interaction data to obtain standardized modal feature data; Topological feature fusion module, used to perform topological data analysis on the preprocessed data and obtain the fused topological feature vector; The causal inference module is used to perform causal relationship analysis between multimodal features, calculate the causal contribution of features and perform optimized fusion; Intent model training module, used to train intelligent customer service intent recognition models based on cross-modal self-supervised contrastive learning methods; The intention recognition and output module is used to achieve accurate multi-level hierarchical recognition and classification output of user intentions based on spectral clustering method.

Images