Method for establishing intelligent video customer service access model based on 5G

Abstract

The application relates to the technical field of 5G video customer service and discloses a 5G-based intelligent video customer service access model establishment method.The method comprises the following steps: collecting real-time video stream data transmitted by a user terminal via a 5G network, extracting multi-dimensional conversation features such as facial expression changes, voice tones and gesture actions, combining historical conversation record intention classification results to build a dynamic intention weight matrix; using an adaptive filtering algorithm to denoise the video stream, extracting key action frames and calculating time sequence correlation; fusing the dynamic intention weight matrix and the time sequence correlation of the key action frames to generate an enhanced user intention vector; based on the matching degree of the vector and historical conversation record intentions, dividing user intention categories through a hierarchical clustering algorithm to establish the mapping relationship between the user intention categories and customer service strategies; and finally generating an access instruction according to the mapping relationship, dynamically allocating optimal customer service resources and adapting to the characteristics of the 5G network.

Description

Technical Field

[0001] This invention relates to the field of 5G video customer service technology, specifically to a method for establishing a 5G-based intelligent video customer service access model. Background Technology

[0002] With the rapid evolution of communication technologies, 5G networks, with their high bandwidth, low latency, and wide connectivity, are driving the upgrade of customer service from traditional voice and text interaction to video interaction. Currently, video customer service systems primarily rely on user-initiated text information or simple voice commands to identify user needs and allocate customer service resources during the access process. However, in real-world applications, the information users transmit through video streams includes not only voice content but also non-verbal information such as facial expressions and gestures. This information is crucial for accurately understanding user intent. Existing video customer service access methods often neglect the extraction and utilization of these multi-dimensional conversational features, generating user intent judgments based solely on a single type of information, resulting in insufficient comprehensiveness and accuracy in intent recognition.

[0003] Existing systems, when processing real-time video stream data, are susceptible to signal fluctuations and noise interference that may occur during 5G network transmission. This can lead to redundant information or distortion in the video frame sequence, making it difficult to effectively extract key action frames and consequently affecting the analysis of the temporal correlation of user behavior. Furthermore, in the intent matching and customer service resource allocation stages, most existing methods rely on fixed rules or static historical data for decision-making. They cannot dynamically adjust intent judgment weights based on changes in real-time user session characteristics, nor can they fully integrate intent classification results from user historical session records to build dynamic correlation models. This results in low matching between intent category classification and customer service strategies, often leading to unreasonable allocation of customer service resources. For example, complex questions are assigned to junior customer service staff, or simple questions occupy senior customer service resources, which not only reduces customer service efficiency but also negatively impacts user experience.

[0004] As users increasingly demand real-time customer service, existing systems lack efficient algorithms in the overall process of feature fusion, intent clustering, and command generation. Insufficient coordination between different stages results in long response times from user-initiated video requests to successful customer service access, failing to meet users' low-latency service needs in 5G network environments. These issues collectively restrict the application and development of intelligent video customer service systems in 5G network environments, necessitating a method for establishing an access model that fully utilizes multi-dimensional session features and dynamically optimizes intent recognition and resource allocation. Summary of the Invention

[0005] The purpose of this invention is to provide a method for establishing a 5G-based intelligent video customer service access model, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides a method for establishing a 5G-based intelligent video customer service access model, the method comprising: Real-time video stream data transmitted by user terminals through a 5G network is collected, and multi-dimensional conversation features are extracted from the video stream data. The multi-dimensional conversation features include facial expression change features, voice tone features, and gesture features. An initial user intent vector is generated based on multi-dimensional session features, and a dynamic intent weight matrix is ​​constructed by combining the intent classification results from historical session records. An adaptive filtering algorithm is used to denoise real-time video stream data, key action frames are extracted from the denoised video frame sequence, and the temporal correlation between key action frames is calculated. The dynamic intent weight matrix and the temporal correlation of key action frames are fused to generate an enhanced user intent vector. Based on the enhanced user intent vector and the intent matching degree of historical session records, a hierarchical clustering algorithm is used to classify user intent categories and establish a mapping relationship between intent categories and customer service strategies. Based on the mapping relationship, video customer service access instructions under the 5G network are generated, and the optimal customer service resources are dynamically allocated.

[0007] Preferably, the extraction of multi-dimensional session features from the video stream data includes: Face region detection is performed on each frame of the video stream data, and the displacement trajectory of facial key points is extracted as facial expression change features; The audio signal in the video stream data is separated, and the spectral energy distribution and fundamental frequency change rate of the audio signal are calculated as speech intonation features. The algorithm uses skeletal keypoint tracking to identify user gestures and extracts the amplitude and frequency of the gestures as gesture features.

[0008] Preferably, constructing the dynamic intent weight matrix includes: Multi-dimensional conversation features are mapped to a preset intent feature space to generate initial feature vectors; Calculate the confidence weight of each dimension of the conversation feature based on the frequency of the same intent classification results in the historical conversation records; The initial feature vector is weighted and combined with the confidence weight to generate a dynamic intent weight matrix.

[0009] Preferably, the extraction of key action frames from the denoised video frame sequence includes: Optical flow analysis is performed on the denoised video frame sequence to calculate the motion vector magnitude between adjacent frames; Based on the detection results of abrupt changes in motion vector amplitude, video frames exceeding a preset threshold are selected as key action frames.

[0010] Preferably, the calculation of the temporal correlation between key action frames includes: Extract the image feature vectors of key action frames and calculate the cosine similarity of the feature vectors of adjacent key action frames; A temporal correlation score is generated based on the decay of cosine similarity and the time interval between key action frames.

[0011] Preferably, the generation of the enhanced user intent vector includes: The dynamic intent weight matrix is ​​decomposed into multiple sub-weight vectors by column; The time-series correlation score is multiplied by the sub-weight vector to generate a weighted intent component. All weighted intent components are normalized and concatenated to generate an enhanced user intent vector.

[0012] Preferably, the step of classifying user intent categories using a hierarchical clustering algorithm includes: The user intent vector input will be enhanced by a pre-defined clustering model, and hierarchical clustering will be performed based on the Euclidean distance between the vectors. Clustering results are evaluated using silhouette coefficients, and the number of clustering layers is dynamically adjusted until a preset clustering quality threshold is met.

[0013] Preferably, establishing the mapping relationship between intent categories and customer service strategies includes: Based on the intent category labels in the clustering results, match the service policy response data in the historical session records; Frequent pattern mining algorithms are used to extract combinations of high-frequency service strategies, generating mapping rules from intent categories to service strategies.

[0014] Preferably, the dynamic allocation of optimal customer service resources includes: Calculate the suitability score for different customer service resources based on the priority of intent categories in the video customer service access instructions; Based on the real-time load status and historical service success rate of customer service resources, select the customer service resources with the highest suitability score.

[0015] Preferably, the calculation of the suitability score for different customer service resources includes: Extract the professional skill tags and intent category processing time from customer service resources; A suitability score is generated based on the matching degree between the priority of intent category and professional skill tags, as well as the inverse weight of processing time.

[0016] Compared with the prior art, the beneficial effects of the present invention are: In terms of multi-dimensional conversation feature extraction, this method overcomes the limitations of existing methods that rely solely on single voice or text information. By collecting real-time video stream data transmitted from user terminals via 5G networks, it simultaneously extracts facial expression change features, voice tone features, and gesture features, achieving comprehensive capture of user conversation information. These multi-dimensional features can reflect the user's true needs and emotional state from different perspectives. For example, changes in facial expressions such as frowning and smiling can help determine the user's level of satisfaction or the urgency of the problem, while gestures such as pointing and gesturing can supplement the explanation of the user's specific demands. Combined with voice tone features, this allows the generation of the initial user intent vector to be more closely aligned with the user's actual situation, reducing misjudgments of intent due to incomplete information.

[0017] Regarding the dynamic optimization of intent recognition, this method constructs a dynamic intent weight matrix by combining intent classification results from historical conversation records. This allows intent judgment to move beyond static data and assign dynamically changing weights to real-time conversation features across different dimensions based on the user's past interaction habits and question types. For example, for users who have repeatedly inquired about technical issues, the weight of keywords in their voice tone can be appropriately increased in subsequent conversations. For users who habitually use gestures to assist in expression, the influence of gesture features on intent vector generation is enhanced. This dynamic adjustment mechanism allows intent recognition to adapt to the interaction characteristics of different users, further improving the targeting and accuracy of intent judgment.

[0018] In the video stream data processing stage, adaptive filtering algorithms are used to denoise the real-time video stream, effectively filtering out noise interference that may exist in 5G network transmission and reducing redundancy and distortion information in the video frame sequence. Key action frames extracted based on this are more representative. By calculating the temporal correlation between key action frames, the dynamic changes in user behavior can be clearly captured, such as the sequence of gestures during a user's description of a problem and the rhythm of facial expressions changing with the speech content. The enhanced user intent vector generated after feature fusion of this temporal information with the dynamic intent weight matrix can simultaneously consider static features and dynamic behavioral correlations, making the expression of user intent more complete and providing richer evidence for subsequent intent classification.

[0019] In terms of intent classification and customer service resource allocation, based on enhancing the intent matching degree of user intent vectors and historical session records, a hierarchical clustering algorithm is used to classify user intent categories. This allows user requests with similar needs and interaction characteristics to be grouped into the same category. Simultaneously, a mapping relationship is established between intent categories and customer service strategies, enabling different types of user needs to be accurately matched with corresponding customer service resources. For example, intent categories involving complex technical issues are mapped to customer service teams with professional technical backgrounds, while simple inquiry intent categories are matched to regular customer service resources. This precise mapping mechanism avoids mismatch and waste of customer service resources. Furthermore, based on the mapping relationship, video customer service access instructions under the 5G network are dynamically generated, enabling rapid response to user requests. Combined with the low latency characteristics of the 5G network, this further shortens customer service access waiting time, improves the interaction efficiency between users and customer service, and enhances the user experience in video customer service.

[0020] This method achieves multi-stage collaborative optimization in its overall process design. From multi-dimensional feature extraction and dynamic weight matrix construction to noise reduction, feature fusion, and hierarchical clustering, each step forms a tightly connected technical chain, ensuring efficient operation throughout the entire process from user video stream data acquisition to customer service resource allocation. This synergy not only improves the stability and reliability of the entire access model but also adapts to the characteristics of large video stream data volume and high transmission speed in the 5G network environment. It provides a feasible path for the large-scale application of intelligent video customer service systems in 5G scenarios, helping to promote the development of the customer service industry towards a more intelligent and efficient direction. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating the working principle of the 5G-based intelligent video customer service access model establishment method described in this invention. Figure 2 A flowchart for extracting multi-dimensional conversation features; Figure 3 A flowchart for generating enhanced user intent vectors. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figure 1This invention provides a method for establishing a 5G-based intelligent video customer service access model. The method includes: collecting real-time video stream data transmitted by user terminals via a 5G network; extracting multi-dimensional conversation features from the video stream data, including facial expression changes, voice tone, and gestures; generating an initial user intent vector based on the multi-dimensional conversation features; constructing a dynamic intent weight matrix by combining intent classification results from historical conversation records; performing noise reduction on the real-time video stream data using an adaptive filtering algorithm; extracting key action frames from the denoised video frame sequence; calculating the temporal correlation between key action frames; fusing the dynamic intent weight matrix with the temporal correlation of key action frames to generate an enhanced user intent vector; classifying user intent categories based on the intent matching degree between the enhanced user intent vector and historical conversation records using a hierarchical clustering algorithm; establishing a mapping relationship between intent categories and customer service strategies; and generating video customer service access instructions under the 5G network based on the mapping relationship, dynamically allocating optimal customer service resources. This method utilizes the high bandwidth and low latency characteristics of the 5G network to ensure rapid transmission and processing of real-time video stream data, thereby improving the response accuracy and efficiency of the customer service system.

[0024] Example 1: See Figure 2 The system receives real-time video stream data from user terminals via a 5G network interface. The video stream uses H.265 encoding to adapt to the high-efficiency transmission characteristics of the 5G network. After protocol parsing, the data packets are sent to the feature extraction module for processing. In the facial expression feature extraction stage, face region detection is performed on each frame of the video stream data. A face detection model based on a convolutional neural network is used to locate the bounding boxes of the facial regions. This model is pre-trained on a dataset containing multiple lighting conditions and has robustness to different skin tones and poses. After detecting the face regions, a facial keypoint detection algorithm is used to identify the coordinate positions of core feature points such as eyebrows, eyes, nose, and lips. By tracking the displacement trajectories of these keypoints in consecutive frames, subtle changes in muscle movement are captured. The displacement trajectory data is smoothed and filtered to eliminate jitter noise, ultimately forming a temporal feature sequence representing facial expression changes. This sequence can reflect the emotional fluctuations of the user during conversation.

[0025] The processing of speech intonation features requires first separating the audio signal from the video stream, then using audio-video demultiplexing technology to extract the audio track from the container format. The audio signal is resampled to a standard sampling rate to ensure feature consistency. Spectral energy distribution analysis uses short-time Fourier transform to convert the time-domain signal into a frequency-domain representation. A spectrogram is constructed by calculating the energy values ​​of each frequency component, and the distribution patterns of energy in specific frequency bands are statistically analyzed to capture timbre characteristics. The calculation of the fundamental frequency variation rate uses an autocorrelation algorithm to detect the vocal cord vibration period. The fundamental frequency fluctuation curve is obtained by measuring the frequency difference between adjacent vibration periods; this curve effectively reflects the speaker's intonation fluctuations and stress patterns. Gesture motion features are acquired through a skeletal keypoint tracking algorithm. This algorithm, based on a lightweight network structure built with deep separable convolution, can identify the three-dimensional coordinates of human joints in real time. The system extracts the spatial positions of key joints such as the hand, elbow, and shoulder from video frames. Gesture amplitude parameters are obtained by calculating the motion vectors of joints between consecutive frames, and the amplitude values ​​are normalized based on the ratio of joint displacement distance to a reference length. Gesture frequency is measured by detecting the repetition intervals of periodic movements. A peak detection algorithm is used to identify the peak and trough positions of the movement sequence, and the movement frequency is calculated based on the time difference between the peaks. These parameters together constitute the dynamic features describing the user's body language.

[0026] In the initial user intent vector generation stage, the aforementioned multi-dimensional conversation features are mapped to a pre-defined intent feature space. This feature space, trained using a large amount of historical conversation data, includes semantic dimensions such as emotional state, consultation intent, and urgency. Facial expression features are mapped to emotion intensity vectors, speech intonation features are converted into tone feature vectors, and gesture features are encoded into body expression vectors. These vectors are concatenated after dimensional alignment to form the initial feature representation. The feature mapping process employs a non-linear transformation function, using neural network layers to learn the complex correspondence between features and the intent space. Intent classification results from historical conversation records are used to calculate feature confidence weights. The system retrieves recently stored similar conversation records and counts the frequency of occurrence of each dimension feature in the correct intent classification. The confidence of facial expression features is calculated based on their matching degree with emotion labels, speech feature weights are determined based on the correlation strength between tone patterns and intent types, and the importance of gesture features is evaluated through their correlation with specific operation commands. The weight calculation uses a sliding window mechanism, prioritizing conversation records with close temporal proximity to ensure the timeliness of the weight parameters. The construction of the dynamic intent weight matrix employs a feature-level fusion strategy, combining the initial feature vector with confidence weights in a weighted manner. The weight allocation is dynamically adjusted based on feature quality. When audio signal quality is poor, the weight of speech features is reduced; when the user's face is obscured, the weight of facial expression features is decreased. This adaptive mechanism ensures system stability even when some features are missing. The weighted combination process introduces feature interaction terms to capture the synergistic effects between different modal features. For example, the synchronicity between facial expression changes and intonation fluctuations may enhance the representation of a specific intent. The final generated dynamic intent weight matrix includes a time dimension, reflecting the evolution trend of user intent during the conversation. The implementation adopts a pipelined architecture design, with the feature extraction module and weight calculation module working in parallel, and data synchronization achieved through a circular buffer. The real-time requirements of 5G network transmission necessitate a multi-threaded processing mechanism in the feature extraction stage, with tasks such as video decoding, face detection, and audio processing allocated to dedicated computing units. Historical data retrieval establishes an inverted index structure, supporting fast similarity matching based on feature hash values, ensuring that weight calculation meets the response requirements of real-time conversations.

[0027] Example 2: The system receives real-time video stream data from the front-end processing module. Although this data is transmitted with low latency via a 5G network, various noises are still introduced during acquisition and transmission, including sensor noise caused by changes in lighting, block artifacts caused by network compression, and motion blur caused by mobile device jitter. An adaptive filtering algorithm is designed to address these mixed noise characteristics. Its core lies in constructing a filter bank that can dynamically adjust parameters. This filter bank includes a spatiotemporal joint denoising module. Spatial denoising uses an edge-preserving smoothing algorithm to process noise in a single frame, while temporal denoising uses inter-frame motion compensation to suppress flicker noise. The adaptive adjustment of filtering parameters is based on real-time analysis of video content characteristics. The system continuously monitors the texture complexity, motion intensity, and brightness change rate of video blocks. For high-texture areas, a weaker smoothing intensity is applied to preserve details, while a stronger denoising intensity is applied to flat areas. For high-speed motion sequences, the weight of temporal filtering is automatically increased, while for static scenes, the emphasis is placed on spatial denoising effects. This adaptive mechanism is achieved through a feedback loop. The system periodically evaluates the quality metrics of the denoised video, such as peak signal-to-noise ratio and structural similarity, and compares them with preset thresholds to automatically adjust filter parameters, ensuring that the best balance between detail preservation and noise suppression can be achieved in different scenarios.

[0028] After noise reduction, the system extracts key motion frames from the video frame sequence, a process that relies on detailed analysis of the video content's motion characteristics. The optical flow analysis algorithm employs a deep learning-based dense optical flow estimation network, pre-trained on a large dataset of synthetic optical flow, capable of accurately calculating the motion vector for each pixel between adjacent frames. The motion vector amplitude is calculated using a two-dimensional vector magnitude formula, generating a heatmap representing the inter-frame motion intensity. Abrupt change detection algorithms scan the motion intensity sequence, searching for peaks with significantly higher motion amplitudes than surrounding frames. These peaks typically correspond to key gestures such as the start, turn, or end of a user's hand gesture. The preset threshold is not fixed but dynamically adjusted based on the overall motion level of the video content. The system calculates the mean and variance of the sequence's motion amplitudes and uses an adaptive thresholding algorithm to ensure accurate capture of meaningful motion changes across videos with varying motion intensities. The selected key motion frames not only contain obvious limb movements but also capture subtle facial expressions, such as rapid blinking or twitching of the mouth. These subtle movements often carry important intent information.

[0029] The calculation of temporal correlation between key action frames requires establishing semantic connections between frames. The system first performs deep feature extraction on each key frame. The feature extraction network uses a convolutional neural network pre-trained on a large image dataset. After removing the final classification layer, the output of the intermediate layer is used as an abstract representation of the frame. This representation includes the semantic content, texture features, and structural information of the image. The cosine similarity calculation of the feature vectors of adjacent key action frames measures their directional consistency in the feature space. High similarity indicates semantic coherence between the two frames, while low similarity suggests significant changes in the scene or action. The generation of temporal correlation scores not only considers simple similarity values ​​but also introduces a time decay factor. The shorter the time interval between adjacent frames, the greater the influence of similarity on their correlation. The longer the time interval, the more naturally the correlation score will decay even if the content is similar. This design can effectively distinguish between natural transitions in continuous actions and abrupt content changes. For example, the inter-frame correlation of a user's continuous waving gesture will be high, while the inter-frame correlation of a sudden change from waving to clenching a fist will be low.

[0030] The entire processing flow adopts a multi-stage pipeline architecture, with a double-buffering mechanism between the noise reduction module and the keyframe extraction module to ensure the continuity of video data processing. The temporal correlation calculation module uses a sliding window strategy, maintaining only the keyframe features within a recent time range, satisfying both real-time requirements and ensuring calculation accuracy. The high bandwidth of the 5G network enables the system to receive high-quality video streams, providing a data foundation for subsequent detailed analysis, while its low latency ensures the timeliness of the entire processing chain from data acquisition to key motion identification. The system establishes an independent processing context for each video session, maintaining the motion feature baseline, noise model parameters, and temporal correlation patterns for that session. This isolation design allows the system to process multiple concurrent video customer service sessions simultaneously without mutual interference. All intermediate processing results, including denoised video frames, optical flow motion maps, keyframe indexes, and correlation scores, are timestamped and stored in a cache database for subsequent feature fusion modules to access as needed. The entire implementation emphasizes the efficiency and adaptability of the algorithm, maintaining stable performance under different network conditions, lighting environments, and user behavior patterns.

[0031] Taking a user consulting a complex business matter through the video customer service function of a mobile banking application as an example, the user is in a large shopping mall with slightly fluctuating network signal. The movement of people in the background and changes in indoor lighting pose challenges to the video stream when using the 5G network for the video call. The user speaks while using gestures to try to express their needs more clearly; for example, when describing the "combination loan" they need, they make a merging gesture with both hands. The system receives real-time video stream data from the user's terminal through the 5G network access point. After decoding, the initial data shows slight block noise and motion blur caused by slight hand tremors. An adaptive filtering algorithm begins to denoise this video stream. The algorithm initializes a set of filter parameters based on the statistical characteristics of the current frame. The spatial domain filter smooths the block noise in a single frame, while the temporal domain filter analyzes the pixel residuals caused by jitter between consecutive frames. During processing, the system detected local lighting changes caused by other people walking in the user's background. The filter parameters were dynamically adjusted accordingly, reducing the smoothing intensity in this affected area to avoid excessive blurring of the user's own image information. Simultaneously, the temporal filtering weight was increased to enhance motion blur in the hand area. After processing, although the background remained, noise and blur were significantly reduced, and the user's outline and movements became clear and stable.

[0032] The denoised video frame sequence is fed into a key motion frame extraction module, which performs frame-by-frame optical flow analysis. The optical flow analysis algorithm calculates the motion vector of each pixel between adjacent frames and generates a heatmap representing the overall motion intensity. When the user speaks calmly, the motion vector amplitude remains at a low level. When the user begins to make the "combination loan" gesture, a significant peak appears in the motion vector amplitude of the hand area. A sudden change point detection algorithm tracks the curve of the motion amplitude across the entire frame in real time. When the detected amplitude exceeds a threshold dynamically calculated based on the motion level of the previous few seconds, the frame is identified as a potential key motion frame. The system does not simply extract the peak point but examines the motion pattern of several frames before and after the peak point to confirm that it is a complete gesture with a start, unfolding, and ending, rather than a random wave. Ultimately, three video frames containing the initial gesture posture, the highest point of the hands merging, and the ending gesture are selected as key motion frames representing the expression. Calculating the temporal correlation between key motion frames requires analyzing the content coherence of these frames. The system performs deep feature extraction on the three selected key frames. The feature extraction network outputs high-dimensional feature vectors for each keyframe. These vectors encode the semantic content of the image. For example, the feature vector of the first frame might highlight the state of "hands separating and preparing," the feature vector of the second frame strongly points to the form of "hands coming together," and the feature vector of the third frame reflects the information of "hands falling back after the gesture is completed." The cosine similarity of the feature vectors of adjacent keyframes is calculated. The similarity between the first and second frames might be moderate because the actions have changed significantly but the main subject remains the same. The similarity between the second and third frames might be higher because both are continuations of the hand-joined state. The generation of the temporal correlation score combines the cosine similarity and the actual time interval between keyframes. Assuming the time interval between the first and second frames is short and the similarity is moderate, and the time interval between the second and third frames is slightly longer but the similarity is high, the system will give a higher temporal correlation score. This indicates that the user's gesture is a fast, coherent, and meaningful unit of expression, rather than several unrelated random actions. This high correlation score will provide important basis for subsequent feature fusion, suggesting that the intention conveyed by this gesture has high completeness and credibility. The entire process is completed within a very short time after the user's gesture, providing crucial temporal action analysis data for the system to accurately understand the user's intention to apply for a "combined loan" using a combination of gestures and language.

[0033] Example 3: See Figure 3The dynamic intent weight matrix is ​​derived from a weighted integration of multi-dimensional conversation features. It includes intent representation information extracted from multiple channels, such as user facial expressions, voice tone, and gestures. The temporal correlation of key action frames reflects the continuity and change patterns of user behavior over time. The purpose of feature fusion is to organically combine these two different types of information. The dynamic intent weight matrix provides the static distribution of user intent in the feature space, while the temporal correlation depicts the dynamic evolution of these intent features over time. The fusion of the two can produce a richer and more robust intent representation vector. The first step in the fusion process is to decompose the dynamic intent weight matrix into multiple sub-weight vectors column-wise. Each column of the matrix corresponds to an intent dimension or feature channel, such as columns representing emotional tendencies, consultation types, and urgency levels. The decomposition process is implemented through a matrix block algorithm, dividing the original matrix into several sub-matrices along the column direction. Each sub-matrix is ​​considered a sub-weight vector, and these vectors together constitute different aspects of the intent representation. The sub-weight vector has the same dimension as the original feature space, and its element values ​​reflect the degree of contribution of the corresponding feature to the specific intent dimension in the current session. The magnitude of the value is positively correlated with the importance of the feature.

[0034] The dot product operation between the temporal correlation score and the sub-weight vector is a core step in feature fusion. Mathematically, the dot product operation is represented as the sum of the element-wise multiplications of two vectors. Specifically, for the i-th sub-weight vector... Time-series correlation score Its dot product operation can be expressed as ,in It is the generated weighted intention component. It is a scalar form of time-series correlation score. This represents the scalar multiplication of a scalar and a vector. The temporal correlation score acts as a dynamic adjustment factor, derived from a quantitative assessment of the correlation between key action frames. A higher score indicates smoother and more consistent action or state transitions between adjacent frames, suggesting the stability or continuity of user intent, thus strengthening the influence of the corresponding sub-weight vector. Conversely, a lower score may indicate a sudden change or interruption in user behavior, suggesting a possible transformation or uncertainty in the hidden meaning, thus weakening its weight. This weighting mechanism allows the generated intent representation to sensitively capture subtle changes in user behavior over time.

[0035] After generating the weighted intent components, the system needs to normalize and concatenate all components. The purpose of normalization is to eliminate potential differences in dimensions and numerical scales between components, ensuring that subsequent clustering or matching operations are not dominated by excessively large values ​​of certain components that could dictate the direction of the entire vector. The normalization process employs an L2 norm-based vector normalization method, dividing each weighted intent component by its own magnitude to transform it into a direction vector. The magnitude is calculated using the Euclidean distance formula. After normalization, all sub-vectors are concatenated into a high-dimensional enhanced user intent vector. The concatenation operation is performed according to the intent dimensions of the sub-vectors to ensure consistency in the vector structure. The dimension of the enhanced user intent vector is equal to the sum of the dimensions of all sub-weight vectors; it preserves the semantic information of the original feature space while incorporating temporal dynamics.

[0036] The entire feature fusion process employs tensor operation optimization in its computational architecture, leveraging the parallel computing capabilities of modern processors to accelerate matrix factorization and dot product operations. The system maintains an independent fusion context for each active customer service session, storing the current session's dynamic intent weight matrix, the latest temporal correlation score queue, and historical fusion results. Fusion operations are triggered at a fixed frequency, typically synchronized with the keyframe extraction frequency of the video stream, ensuring that updates to user intent reflect the latest interaction state in a timely manner. The generated enhanced user intent vector is fed into a first-in-first-out buffer for subsequent hierarchical clustering modules to consume. This buffer design avoids direct coupling between processing modules, improving the system's fault tolerance and scalability. During vector generation, the system records the source weight and correlation score of each weighted component. This metadata is used for subsequent error tracking and model optimization. When intent recognition deviates, it allows for retrospective investigation of which feature channel or time point caused the problem. The parameters of the fusion algorithm, such as the choice of normalization method and the definition of the splicing order, are determined through offline hyperparameter tuning. The tuning process uses historical session data labeled with intent tags, with clustering purity or classification accuracy as the optimization objective. In actual deployment, the system continuously monitors certain statistical characteristics of the fused vectors, such as vector sparsity and magnitude distribution. Significant deviations from historical normal ranges may indicate anomalies in front-end feature extraction, thereby triggering the system's self-checking alarm mechanism. The entire implementation process emphasizes computational efficiency and numerical stability. Especially when dealing with high-dimensional vectors, numerical precision control and rounding error management are employed to ensure consistency of computational results across different hardware platforms.

[0037] Example 4: The system receives enhanced user intent vectors generated by the feature fusion module. These vectors are points in a high-dimensional space, each representing an abstract intent representation of a user interaction session. The hierarchical clustering algorithm adopts a bottom-up aggregation strategy. In the initial stage, each enhanced user intent vector is treated as an independent cluster. The Euclidean distance between all pairs of vectors is calculated to construct a distance matrix. The Euclidean distance measures the straight-line distance between vectors in the feature space; the smaller the value, the more similar the intents of the two sessions. In each iteration, the algorithm finds the two closest clusters and merges them to form a new, larger cluster. After merging, the distances between the new cluster and other clusters are updated. The distance update uses the WARD method, which minimizes the variance increment of the merged cluster, thus ensuring high cohesion of the generated clusters. The clustering process generates a tree-like hierarchy graph, where the leaf nodes are individual intent vectors and the root node is a single cluster containing all vectors. By cutting the hierarchy graph at different heights, clustering results of different granularities can be obtained.

[0038] The silhouette coefficient is used to evaluate the quality of clustering results. This coefficient comprehensively considers both intra-cluster compactness and inter-cluster separation. For a given intent vector, the average distance between it and other vectors in the same cluster is calculated as its intra-cluster dissimilarity. Simultaneously, the average distance from this vector to all vectors in every other cluster is calculated, and the minimum of these is selected as the inter-cluster dissimilarity. The silhouette coefficient of this vector equals the inter-cluster dissimilarity minus the intra-cluster dissimilarity, divided by the larger of the two. The coefficient ranges from -1 to 1; a value closer to 1 indicates a more reasonable vector assignment. The silhouette coefficient of the entire clustering result is the average of the silhouette coefficients of all vectors. The system sets a clustering quality threshold and calculates the silhouette coefficients corresponding to different numbers of clusters by dynamically adjusting the cut height of the phylogenetic graph until a clustering scheme is found that results in an average silhouette coefficient higher than the preset threshold. After clustering, each cluster is assigned an intent category label, generated based on the centroid features of the intra-cluster vectors. The system then performs a matching query in the historical session record database. The historical session records store the enhanced user intent vectors of past sessions, the finally confirmed user intent tags, and the sequence of service strategies actually adopted by the customer service representative. For each newly generated intent category cluster, the system retrieves all session records that fall within the space of that cluster in history, and extracts the service strategy data adopted by the customer service representative in these records. These strategies may include specific operations such as transferring to an expert, sending a knowledge base link, initiating screen sharing, and enabling multilingual translation.

[0039] Frequent pattern mining algorithms are used to discover patterns in large amounts of historical policy data. The Apriori algorithm discovers frequent itemsets between policies through a layer-by-layer iterative search method. The algorithm first counts the frequency of each individual service policy, selecting policies with a frequency greater than a minimum support threshold to form frequent 1-itemsets. Then, based on the previous frequent itemsets, candidate next-order itemsets are generated. The database is then scanned to count the frequency of these candidate itemsets, selecting itemsets that meet the minimum support threshold. This process iterates until no new frequent itemsets can be generated. For example, when processing historical policy data for the intent category "billing query," the algorithm might find that the tuple {send billing query link, verify user identity} frequently appears simultaneously. See Table 1, which shows a simplified mapping between intent categories and frequent service policy patterns.

[0040] Table 1: Mapping Table of Intent Categories and Service Policies

[0041] Based on the frequently identified policy combinations, the system generates mapping rules from intent categories to service policies. These rules are represented using an "IF-THEN" production rule. For example, if the intent category is "fault reporting," then "THEN" will recommend the policies "collect device information," "initiate remote diagnostics," and "generate a work order" with high probability. Each mapping rule is accompanied by support and confidence metrics. Support reflects the prevalence of the rule in historical data, while confidence reflects the reliability of the conclusion when the current conditions are met. These rules are structured and stored in a policy knowledge base, forming a decision tree or a rule set. When a new user session is classified into a certain intent category, the system can quickly retrieve the corresponding recommended policy sequence from the knowledge base, providing operational guidance for customer service personnel. The entire mapping relationship is established through a dynamic update process. New successful session records are continuously added to the historical database, periodically triggering a new round of frequent pattern mining, enabling the mapping rules to adapt to the evolution of business strategies and changes in user behavior patterns.

[0042] Example 5: After the system completes the classification of user intent categories and determines the corresponding service strategies, it generates a structured video customer service access instruction. This instruction encapsulates the unique identifier of the session, the identified intent category code, the recommended service strategy list, and the intent priority identifier. The core objective of the dynamic allocation mechanism is to route the current session to the most suitable customer service agent. This decision is based on a comprehensive quantitative evaluation of multi-dimensional adaptation factors. Calculating the adaptation score of different customer service resources is the first step in the allocation decision. The system needs to extract various attribute data of the target agent from the customer service resource management database. Professional skill tags are structured data describing the agent's business capabilities, usually existing in the form of a tag cloud. For example, an agent may have tags such as "international business," "credit card service," and "troubleshooting." These tags are associated with their training experience, certifications, and past handling cases. Historical service records provide actual performance data of the agents. The system pays particular attention to the historical data of agents handling sessions of specific intent categories, including average processing time, resolution rate, and customer satisfaction score. The priority of intent categories is set by the previous analysis module, and is usually determined based on factors such as urgency, business value, or customer level. For example, the priority of intents such as "account theft" will be much higher than that of intents such as "product consultation".

[0043] The fit score calculation requires matching the agent's skill tags with the priority requirements of the current intent, and its quantitative relationship can be expressed as follows: ;in: This represents the final calculated fit score. The higher the score, the better the match between the agent and the current session. This is the total number of business skill tags associated with the current intent category. The system retrieves the set of skills that need to be matched from the knowledge base based on the intent category. It is the first The weight coefficient of each skill tag is positively correlated with the priority of the intent category. The higher the priority of the intent, the greater the weight assigned to its core skill tag. It is a matching indicator function, when the agent has the first The value is 1 when the skill tag is active, and 0 otherwise. This function ensures that only skills actually mastered by the agent contribute to the score. It is an adjustment coefficient used to balance the influence of skill matching and processing efficiency on the total score. Its value is determined through retrospective analysis of historical allocation effects. This is the historical average processing time for this agent to handle similar current requests. The shorter the processing time, the higher the score for this item, reflecting the consideration of processing efficiency.

[0044] After calculating the initial suitability score, the system needs to refine it based on the real-time load status of customer service resources. Real-time load status is continuously acquired through heartbeats and status signaling from the agent console, including the number of concurrent sessions currently being processed, session duration, and whether the agent is in a busy or short break state. The system defines a real-time load coefficient for each agent. This coefficient is positively correlated with the number of concurrent sessions. When the number of sessions handled by an agent exceeds its rated processing capacity, the load coefficient increases significantly. The corrected suitability score is the original score divided by the load coefficient, thus avoiding assigning new sessions to already overloaded agents and achieving load balancing. Historical service success rate is also an important correction factor. The system queries the agent's historical success records when handling similar intents; a higher success rate results in a bonus to the agent's score. Selecting the agent with the highest suitability score is not a simple sorting and taking the maximum value; the system considers the long-term effects of the allocation strategy. To prevent the few agents with the highest scores from always being under high load, the system introduces a polling mechanism and fatigue protection. When an agent continuously handles multiple high-priority sessions, their suitability score is manually lowered, giving other agents with similar capabilities the opportunity to take orders. The allocation decision also needs to consider the continuity requirements of the session. If the current session is a continuation of a previous incomplete session, the system will prioritize assigning the session to the original agent to maintain service continuity and contextual understanding. The final allocation instruction is sent to the agent's workstation via a video customer service control channel over a 5G network. The agent receives a prompt interface containing basic user information, the identified intent, and the recommendation strategy, and simultaneously establishes an end-to-end video connection with the user's terminal. The entire allocation process is completed within milliseconds, with virtually no waiting time perceptible to the user. The system records detailed decision logs for each allocation, including rating details for all candidate agents and the reasons for the final selection. These logs are used to periodically optimize the parameters and weight settings in the rating model.

[0045] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0046] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for establishing a 5G-based intelligent video customer service access model, characterized in that, Includes the following steps: Real-time video stream data transmitted by user terminals through a 5G network is collected, and multi-dimensional conversation features are extracted from the video stream data. The multi-dimensional conversation features include facial expression change features, voice tone features, and gesture features. An initial user intent vector is generated based on multi-dimensional session features, and a dynamic intent weight matrix is ​​constructed by combining the intent classification results from historical session records. An adaptive filtering algorithm is used to denoise real-time video stream data, key action frames are extracted from the denoised video frame sequence, and the temporal correlation between key action frames is calculated. The dynamic intent weight matrix and the temporal correlation of key action frames are fused to generate an enhanced user intent vector. Based on the enhanced user intent vector and the intent matching degree of historical session records, a hierarchical clustering algorithm is used to classify user intent categories and establish a mapping relationship between intent categories and customer service strategies. Generate video customer service access instructions under the 5G network based on the mapping relationship, and dynamically allocate the optimal customer service resources; The step of extracting key action frames from the denoised video frame sequence and calculating the temporal correlation between key action frames includes: Optical flow analysis is performed on the denoised video frame sequence to calculate the motion vector magnitude between adjacent frames; Based on the detection results of abrupt changes in motion vector amplitude, video frames exceeding a preset threshold are selected as key action frames. Extract the image feature vectors of key action frames and calculate the cosine similarity of the feature vectors of adjacent key action frames; Temporal correlation is generated based on the decay degree of cosine similarity and the time interval of key action frames; The generation of the enhanced user intent vector includes: The dynamic intent weight matrix is ​​decomposed into multiple sub-weight vectors by column; The temporal correlation degree is multiplied by the sub-weight vector to generate a weighted intention component. All weighted intent components are normalized and concatenated to generate an enhanced user intent vector.

2. The method for establishing a 5G-based intelligent video customer service access model according to claim 1, characterized in that, The extraction of multi-dimensional session features from video stream data includes: Face region detection is performed on each frame of the video stream data, and the displacement trajectory of facial key points is extracted as facial expression change features; The audio signal in the video stream data is separated, and the spectral energy distribution and fundamental frequency change rate of the audio signal are calculated as speech intonation features. The algorithm uses skeletal keypoint tracking to identify user gestures and extracts the amplitude and frequency of the gestures as gesture features.

3. The method for establishing a 5G-based intelligent video customer service access model according to claim 1, characterized in that, The construction of the dynamic intent weight matrix includes: Multi-dimensional conversation features are mapped to a preset intent feature space to generate initial feature vectors; Calculate the confidence weight of each dimension of the conversation feature based on the frequency of the same intent classification results in the historical conversation records; The initial feature vector is weighted and combined with the confidence weight to generate a dynamic intent weight matrix.

4. The method for establishing a 5G-based intelligent video customer service access model according to claim 1, characterized in that, The process of classifying user intent categories using a hierarchical clustering algorithm includes: The user intent vector input will be enhanced by a pre-defined clustering model, and hierarchical clustering will be performed based on the Euclidean distance between the vectors. Clustering results are evaluated using silhouette coefficients, and the number of clustering layers is dynamically adjusted until a preset clustering quality threshold is met.

5. The method for establishing a 5G-based intelligent video customer service access model according to claim 4, characterized in that, The establishment of the mapping relationship between intent categories and customer service strategies includes: Based on the intent category labels in the clustering results, match the service policy response data in the historical session records; Frequent pattern mining algorithms are used to extract combinations of high-frequency service strategies, generating mapping rules from intent categories to service strategies.

6. The method for establishing a 5G-based intelligent video customer service access model according to claim 1, characterized in that, The dynamic allocation of optimal customer service resources includes: Calculate the suitability score for different customer service resources based on the priority of intent categories in the video customer service access instructions; Based on the real-time load status and historical service success rate of customer service resources, select the customer service resources with the highest suitability score.

7. The method for establishing a 5G-based intelligent video customer service access model according to claim 6, characterized in that, The calculation of the suitability score for different customer service resources includes: Extract the professional skill tags and intent category processing time from customer service resources; A suitability score is generated based on the matching degree between the priority of intent category and professional skill tags, as well as the inverse weight of processing time.

Images