Hotel guest room multi-mode intelligent customer service interaction system based on voice gateway

Abstract

The invention discloses a hotel guest room multi-mode intelligent customer service interaction system based on a voice gateway, and particularly relates to the technical field of multi-mode intelligent customer service interaction.The hotel guest room multi-mode intelligent customer service interaction system comprises the steps that multi-mode signals are synchronously collected, and confidence marks are given; by dynamically quantifying a modal intervention risk caused by automatic weight-down lag, an automatic weight-down delay coefficient is constructed as a core characterization index for real-time evaluation in the system; and in combination with an abnormal rise coefficient of the computing resource, the actual impact of the low confidence coefficient mode on the computing resource in a drop-weight lagging window period can be accurately evaluated, a resource obstructive linkage imbalance risk evaluation model is constructed, and a resource obstructive linkage imbalance risk index representing the system operation health degree is output. The real-time evaluation of the potential blocking risk in the system is realized; and finally, comparing the resource obstructive chain imbalance risk index with a preset resource obstructive chain imbalance risk index threshold value, and triggering graded early warning, thereby realizing efficient frequency reduction current limiting, task stripping and main mode protection.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal intelligent customer service interaction. More specifically, the present invention relates to a hotel guest room multimodal intelligent customer service interaction system based on a voice gateway. Background Art

[0002] With the continuous development of intelligent speech recognition, computer vision, and multimodal fusion technologies, the intelligent customer service system in hotel guest rooms is gradually shifting from single - voice interaction to a multimodal perception and response mechanism that integrates voice, images, sensors, etc. Among them, the hotel guest room multimodal intelligent customer service interaction system based on a voice gateway can effectively integrate heterogeneous signal sources such as indoor voice input, image information, action recognition, temperature and humidity sensors, improve the accuracy of user instruction recognition and the naturalness of the interaction experience, and become one of the important directions for hotel intelligent upgrading. However, in the process of multimodal input fusion, due to significant differences in the quality, confidence level, and time synchronization of various modal data, the system needs to dynamically down - weight the modal data with a lower confidence level to avoid interfering with the overall reasoning and judgment. In current technologies, the determination of modal confidence and the down - weight operation often have response delays. Especially when the input frequency of weak modal data is high or there are abnormal fluctuations, the down - weight mechanism fails to intervene in time, resulting in the system still continuously processing data tasks with high computational overhead and low effectiveness. More critically, within the time window of lagging down - weight response, the low - confidence modality still occupies a large amount of system computing resources, causing a mismatch in the scheduling priority of core resources, further triggering a series of chain reactions such as modal fusion blockage, semantic recognition deviation, and task response delay, forming the so - called resource - blocking chain imbalance risk. This problem not only weakens the system's response ability to high - confidence main modalities (such as voice commands), but also may lead to a break in the user experience and a decline in service trust.

[0003] Therefore, there is an urgent need to propose an optimization mechanism for multimodal interaction systems for voice gateway architectures that can achieve high - responsiveness dynamic down - weighting and resource allocation control for low - confidence modal data to build an intelligent customer service interaction solution with high robustness, high concurrent bearing capacity, and high user satisfaction. Summary of the Invention

[0004] In order to overcome the above - mentioned defects of the prior art, an embodiment of the present invention provides a hotel guest room multimodal intelligent customer service interaction system based on a voice gateway to solve the problems raised in the above background art.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] The hotel guest room multimodal intelligent customer service interaction system based on a voice gateway includes a multimodal perception annotation module, a down - weight delay analysis module, a resource anomaly analysis module, a resource imbalance risk assessment module, and a risk response regulation module;

[0007] A multi-modal perception annotation module, which is used to collect modal signals of users through multi-source perception terminals deployed in guest rooms to form a multi-modal perception input set, mark the confidence of each modality, and perform weighted scheduling on the marked low-confidence modalities;

[0008] A weighted delay analysis module, which is used to obtain the automatic weighted delay information of low-confidence modal data and obtain the automatic weighted delay coefficient, and analyze the degree of automatic weighted delay of low-confidence modal data;

[0009] A resource anomaly analysis module, which is used to obtain the information of abnormal increase in computing resources during the automatic weighted delay period of low-confidence modal data and obtain the coefficient of abnormal increase in computing resources, and analyze the degree of abnormal increase in computing resources of low-confidence modal data;

[0010] A resource imbalance risk assessment module, which is used to construct a resource blocking chain imbalance risk assessment model according to the automatic weighted delay coefficient and the coefficient of abnormal increase in computing resources, output the resource blocking chain imbalance risk index, and assess the resource blocking chain imbalance risk of the existing intelligent customer service interaction system;

[0011] A risk response regulation module, which is used to compare the resource blocking chain imbalance risk index with a preset resource blocking chain imbalance risk index threshold, and perform early warning grading on the resource blocking chain imbalance risk of the existing intelligent customer service interaction system.

[0012] In a preferred embodiment, the acquisition logic of the automatic weighted delay coefficient is as follows:

[0013] Construct a modal response path graph Gl=(V,E) for the fundamental modal processing, where V is the set of each sub-node of the modal processing, and E is the set of edges, representing the operation jump between nodes. Each edge by mn corresponds to a modal processing jump, and obtain its processing time consumption chs from the system historical log mn ;

[0014] Calculate the total modal processing response time: zxt=Σ (m,n)∈LJ chs mn , where zxt is the total modal processing response time, and LJ is the actual path experienced by the current modality in the modal response path graph;

[0015] Calculate the modal timing drift entropy: Hshift=-Σ (m,n)∈LJ gl mn *log(gl mn ), where Hshift is the modal timing drift entropy, and gl mn is the timing distribution probability of the processing time consumption between node m and node n,

[0016] Define the weight perturbation response function qzr(t) to characterize the dynamic behavior of the modal weight change during the downweighting process. If the system completes the adjustment of the modal weight from t m to t n , then the weight perturbation response function qzr(t) is expressed as where qz init is the modal weight before adjustment, qz(t) is the modal weight at time t, and t ∈ [t m , t n ; Calculate the weight perturbation response value: where Rd is the weight perturbation response value;

[0017] The calculation expression of the automatic downweighting delay coefficient is as follows: zdj = c1*zxt + c2*Hshift + c3*Rd, where zdj is the automatic downweighting delay coefficient, and c1, c2, and c3 are the preset proportionality coefficients of the total modal processing response time, modal timing drift entropy, and weight perturbation response value respectively, and c1, c2, and c3 are all greater than 0.

[0018] In a preferred embodiment, compare the automatic downweighting delay coefficient with the preset automatic downweighting delay coefficient threshold. If the automatic downweighting delay coefficient is greater than the automatic downweighting delay coefficient threshold, generate a downweighting delay signal and mark the downweighting time period of this mode as the downweighting delay time period.

[0019] In a preferred embodiment, the acquisition logic of the calculation resource abnormal rise coefficient is as follows:

[0020] Sample the resource items consumed by running tasks on low-confidence modal data within the automatic downweighting delay time interval [ts, te]. The resource items include the CPU resource utilization rate R1(t), the memory resource utilization rate R2(t), the I / O resource utilization rate R3(t), and the thread resource utilization rate R4(t), and construct a resource matrix;

[0021] Compare with the stable baseline time period [ts - T0, ts] before automatic downweighting, and calculate the relative perturbation amplitude: where ΔR h (t) is the relative perturbation amplitude of the h-th resource, h = {1, 2, 3, 4}, R h (t) is the specific sampling value of the h-th resource, is the average value of the h-th resource utilization rate within the stable baseline time period before automatic downweighting; Calculate the sliding change rate of each resource within the sliding window CK: where VH h is the sliding change rate of the h-th resource, ΔR h(t - k) is the relative perturbation amplitude calculated for the h-th resource at time (t - k), ΔR h (t - k - 1) is the relative perturbation amplitude calculated for the h-th resource at time (t - k - 1);

[0022] Calculate the correlation coefficient between each pair of resources based on the dynamic covariance of the relative perturbation amplitude: where Corr hg (t) is the correlation coefficient between the h-th resource and the g-th resource, ΔR h (τ) is the relative perturbation amplitude calculated for the h-th resource at time τ, is the mean value of the relative perturbation amplitude calculated for the h-th resource within the sliding window CK, ΔR g (τ) is the relative perturbation amplitude calculated for the g-th resource at time τ, is the mean value of the relative perturbation amplitude calculated for the g-th resource within the sliding window CK;

[0023] Calculate the coupling degree between each pair of resources: where oh hg is the coupling degree between the h-th resource and the g-th resource, VH g is the sliding change rate of the g-th resource;

[0024] Calculate the resource fluctuation perception value of the h-th resource: where gz h (t) is the resource fluctuation perception value of the h-th resource, ΔR g (t) is the relative perturbation amplitude calculated for the g-th resource at time t;

[0025] The calculation expression for calculating the resource abnormal rise coefficient is as follows: where jsz is the calculated resource abnormal rise coefficient.

[0026] In a preferred embodiment, construct a resource blocking chain imbalance risk assessment model based on the automatic downgrading delay coefficient and the calculated resource abnormal rise coefficient, and output the resource blocking chain imbalance risk index. The formula on which the resource blocking chain imbalance risk assessment model is based is as follows: zyls = w1 * zdj + w2 * jsz, where zyls is the resource blocking chain imbalance risk index, zdj is the automatic downgrading delay coefficient, jsz is the calculated resource abnormal rise coefficient, w1 and w2 respectively represent the preset proportionality coefficients of the automatic downgrading delay coefficient and the calculated resource abnormal rise coefficient, and both w1 and w2 are greater than 0.

[0027] In a preferred embodiment, the resource blocking chain imbalance risk index is compared with a preset resource blocking chain imbalance risk index threshold to give an early warning and classification of the resource blocking chain imbalance risk of the existing intelligent customer service interaction system, which is specifically as follows:

[0028] If the resource blocking chain imbalance risk index is greater than the resource blocking chain imbalance risk index threshold, a red warning signal is generated and the one-key call mechanism is activated;

[0029] If the resource blocking chain imbalance risk index is less than or equal to the resource blocking chain imbalance risk index threshold, a green warning signal is generated.

[0030] Technical effects and advantages of the present invention:

[0031] 1. Through the multi-modal perception annotation module of the present invention, multi-modal signals are synchronously collected and confidence marks are assigned, realizing the precise identification and prior weight reduction control of low-confidence modal data, and cutting off the interference source of redundant modalities to the main task path in advance; through the weight reduction delay analysis module, the modal intervention risk caused by the lag of automatic weight reduction is dynamically quantified, and an automatic weight reduction delay coefficient is constructed as the core characterization index for real-time evaluation within the system; combined with the calculation resource anomaly increase coefficient introduced in the resource anomaly analysis module, the actual impact of low-confidence modalities on computing resources during the weight reduction lag window period can be accurately evaluated, so as to comprehensively quantify the modal-resource conflict dynamics; further, by integrating the above two core indicators, a resource blocking chain imbalance risk assessment model is constructed, and a resource blocking chain imbalance risk index representing the health of the system operation is output, realizing the real-time assessment of potential blocking risks in the system; finally, the risk response regulation module compares the resource blocking chain imbalance risk index with the preset resource blocking chain imbalance risk index threshold, triggers hierarchical early warning, realizes efficient frequency reduction and flow limiting, task stripping and main modal protection, ensures that the high-confidence main modality obtains a stable computing path and makes a feedforward response to potential congestion risks, not only greatly reducing the resource conflict and semantic deviation risks during multi-modal fusion, but also significantly shortening the system response delay and improving the accuracy and user satisfaction of multi-modal interaction. Description of the Drawings

[0032] For the convenience of those skilled in the art to understand, the present invention will be further described below in conjunction with the drawings;

[0033] Figure 1 It is a flowchart of the system according to the embodiment of the present invention. Detailed Embodiments

[0034] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0035] Embodiment: The present invention provides a hotel guest room multi-modal intelligent customer service interaction system based on a voice gateway as shown in Figure 1 Figure 5, including a multi-modal perception annotation module, a weight reduction delay analysis module, a resource anomaly analysis module, a resource imbalance risk assessment module, and a risk response regulation module;

[0036] The multi-modal perception annotation module is used to collect the modal signals of users through multi-source perception terminals deployed in the guest rooms to form a multi-modal perception input set, mark the confidence of each modality, and perform weight reduction scheduling on the marked modalities with low confidence;

[0037] The weight reduction delay analysis module is used to obtain the automatic weight reduction delay information of the low-confidence modality data and obtain the automatic weight reduction delay coefficient, and analyze the degree of automatic weight reduction delay of the low-confidence modality data;

[0038] The resource anomaly analysis module is used to obtain the information of abnormal increase in computing resources during the automatic weight reduction delay period of the low-confidence modality data and obtain the abnormal increase coefficient of computing resources, and analyze the degree of abnormal increase in computing resources of the low-confidence modality data;

[0039] The resource imbalance risk assessment module is used to construct a resource blocking chain imbalance risk assessment model according to the automatic weight reduction delay coefficient and the abnormal increase coefficient of computing resources, output a resource blocking chain imbalance risk index, and evaluate the resource blocking chain imbalance risk of the existing intelligent customer service interaction system;

[0040] The risk response regulation module is used to compare the resource blocking chain imbalance risk index with a preset resource blocking chain imbalance risk index threshold, and perform early warning grading on the resource blocking chain imbalance risk of the existing intelligent customer service interaction system;

[0041] The multi-modal perception annotation module is used to collect the modal signals of users through multi-source perception terminals deployed in the guest rooms to form a multi-modal perception input set: Among them, is the original input of the i-th modality (including voice modality, visual modality, touch modality, environmental modality, etc.) at time t; feature extraction is performed on each modality: Among them is the feature extraction vector of the i-th modality, hs i(·) is the feature extraction function for the i-th modality (such as MFCC extraction, convolutional encoder, etc.);

[0042] Assign a time-varying confidence function to each modality: ZX i (t) = a1 * QX i + a2 * YZ i - a3 * ZS i + a4 * WE i , where ZX i (t) is the confidence of the i-th modality, QX i is the feature clarity score of the i-th modality, YZ i is the modality context consistency of the i-th modality, ZS i is the noise interference degree of the i-th modality, WE i is the historical performance stability of the i-th modality, and a1, a2, a3, a4 respectively represent the preset proportional coefficients of the feature clarity score, modality context consistency, noise interference degree, and historical performance stability, and a1, a2, a3, a4 are all greater than 0;

[0043] It should be noted that a1, a2, a3, a4 are set according to the actual situation. For example, the expert weighting method is adopted, that is, experts in related fields are invited to determine the preset proportional coefficients of each index through professional opinion surveys and comprehensive evaluations. For example, a1, a2, a3, a4 can be 0.3, 0.2, 0.2, 0.3;

[0044] The feature clarity score is used to measure the structural integrity and recognizability of the current modality signal, and the calculation expression is as follows: where μz j is the mean value of the j-th feature component within the historical time window T, σz j is the standard deviation of the j-th feature component within the historical window T, J is the dimension of the feature vector, and ∈ is a very small constant to prevent division by zero (generally take ∈ = 10 -6 );

[0045] The modality context consistency is used to evaluate the matching degree between the current modality input and the temporal context, and the calculation expression is as follows: where is the feature extraction vector of the i-th modality at time t, is the feature extraction vector of the i-th modality at time t ' , sim(·,·) is the cosine similarity between the feature extraction vectors and , and T is the historical time window;

[0046] The noise interference degree is a reverse index used to evaluate the degree of interference of the current mode (i.e., the larger the value, the worse the quality), and the calculation expression is as follows: where SNR i is the signal-to-noise ratio of the i-th mode, and ∈ is a very small constant to prevent division by zero (generally, ∈ = 10 -6 );

[0047] The historical performance stability is used to measure whether there is a continuous output state with high confidence within the past window for this mode, and the calculation expression is as follows: where ZX i (t”) is the confidence of the i-th mode at time t”, ZXY is the preset confidence threshold, T is the historical time window, and zhs[·] is an indicator function. When the condition ZX i (t”) > ZXY is satisfied, the value of the indicator function is 1;

[0048] It should be noted that the above formulas are all dimensionless and take their numerical values for calculation. Common methods for removing dimensions include Min-Max normalization, Z-Score standardization, etc., which will not be elaborated here;

[0049] Compare the confidence of the i-th mode with the preset confidence threshold, mark the confidence of each mode, and at the same time perform a weighted-down scheduling on the modes with low confidence that have been marked, as follows:

[0050] If the confidence of the i-th mode is less than or equal to the confidence threshold, then mark the i-th mode as a low-confidence mode, and at the same time perform a weighted-down scheduling on the i-th mode;

[0051] The weighted-down delay analysis module is used to obtain the automatic weighted-down delay information of the low-confidence mode data and obtain the automatic weighted-down delay coefficient, and analyze the degree of automatic weighted-down delay of the low-confidence mode data;

[0052] In the present invention, the automatic demotion delay coefficient is an index used to measure the degree of reaction lag in the process of multi-modal intelligent customer service interaction, where the system identifies low-confidence modal signals and completes the demotion process. It reflects the system processing delay experienced from the identification of a low-confidence modality to the actual reduction of its participation weight. It not only reflects the linkage efficiency between modal identification and resource scheduling but also indirectly reveals the dynamic response ability of the intelligent customer service system during the resource allocation process. The larger the automatic demotion delay coefficient, the slower the system's reaction when faced with low-quality input, and there is a significant lag in modal weight adjustment, which may lead to low-quality modalities continuously occupying high computing resources for a long time, thus causing problems such as resource scheduling blockage and abnormal service response. Conversely, the smaller the automatic demotion delay coefficient, the faster the system can complete the identification and demotion regulation of low-confidence modalities, achieving a fast closed-loop of the perception-response-scheduling chain, thereby effectively ensuring the dynamic and reasonable allocation of resources and the stability of customer service interaction. In actual operation scenarios, user input often occurs in a complex dynamic environment. For example, strong background noise in the voice modality or occlusion or blurring in the image modality will lead to a decrease in the confidence of the perceived modality. If the system cannot promptly demote such "low-confidence modalities", the system resources will be occupied by incorrect modalities for a long time, preventing subsequent high-quality modal signals from participating in decision-making calculations in a timely manner, resulting in interaction blockage. The automatic demotion delay coefficient proposed in the present invention can be used as a key performance parameter for dynamically monitoring the reaction ability of the intelligent interaction system. Its design not only realizes the quantitative modeling of "delay behavior" but also provides an explicit constraint index for resource scheduling. The introduction of the automatic demotion delay coefficient enables the intelligent customer service system to have the ability to self-detect and respond to "delay anomalies" in the modal regulation chain. By calculating and analyzing this coefficient in real time, the system can quickly identify potential hidden delay risks that may cause resource blockage in the interaction process. Through joint analysis with the abnormal increase coefficient of computing resources, a risk assessment model for resource-blocking chain imbalance is constructed, and finally, a resource-blocking chain imbalance risk index is output, helping the system to complete pre-perception and hierarchical response before interaction jams and service failures occur, thereby significantly improving the robustness and service stability of the system. The introduction of the automatic demotion delay coefficient also enhances the system's ability to sensitively capture the changing trends of the quality of different modal signals, making the collaborative scheduling between modalities more flexible and intelligent, and greatly improving the dynamic allocation efficiency of resources. Through the coordinated operation of the automatic demotion delay coefficient and the abnormal increase coefficient of computing resources, the system can maintain high interaction accuracy while significantly reducing the probability of interaction failures caused by "delay + resource abuse", especially suitable for intelligent customer service tasks in complex scenarios such as multi-turn conversations, asynchronous modal inputs, and sudden state interferences, with broad engineering application value and practical deployment significance. The automatic demotion delay coefficient is not only an important index for measuring the processing response efficiency of low-confidence modalities but also the core basis for constructing a risk assessment model for resource-blocking chain imbalance.Its introduction effectively solves the problem of the inability to dynamically quantify the modal processing delay in traditional interactive systems, significantly enhances the system's ability to pre-identify and intervene in abnormal resource states, and is an important technical support for realizing stable, efficient, and intelligent customer service interactions.

[0053] The acquisition logic of the automatic demotion delay coefficient is as follows:

[0054] Construct a modal response path graph Gl=(V, E) based on multiple intermediate paths of fundamental modal processing (such as judgment, scheduling, caching, demotion), where V is the set of each sub-node of modal processing (including judgment nodes, scheduling nodes, caching nodes, demotion nodes, etc.), and E is the set of edges, representing the operation jumps between nodes. Each edge by mn corresponds to a modal processing jump, and obtain its processing time consumption chs from the system historical log mn ;

[0055] Calculate the total modal processing response time: zxt = Σ (m,n)∈LJ chs mn , where zxt is the total modal processing response time, and LJ is the actual path experienced by the current mode in the modal response path graph;

[0056] Analyze whether there is a drift in the path time distribution during the process of the mode from marking low confidence to triggering actual demotion, and calculate the modal timing drift entropy: Hshift = -Σ (m,n)∈LJ gl mn *log(gl mn ), where Hshift is the modal timing drift entropy, and gl mn is the timing distribution probability of the processing time consumption between node m and node n,

[0057] Define the weight perturbation response function qzr(t) to characterize the dynamic behavior of the modal weight change during the demotion process. If the system completes the adjustment of the modal weight from t m to t n , then the weight perturbation response function qzr(t) is expressed as where qz init is the modal weight before adjustment, qz(t) is the modal weight at time t, and t ∈ [t m , t n ; Calculate the weight perturbation response value: where Rd is the weight perturbation response value;

[0058] The calculation expression of the automatic downweighting delay coefficient is as follows: zdj = c1 * zxt + c2 * Hshift + c3 * Rd, where zdj is the automatic downweighting delay coefficient, and c1, c2, and c3 are the preset proportionality coefficients of the total response time of modal processing, the entropy of modal timing drift, and the response value of weight perturbation, respectively, and c1, c2, and c3 are all greater than 0;

[0059] It should be noted that the above formulas are all calculated by taking the numerical values after dimensionless processing. Common methods for removing dimensions include Min-Max normalization, Z-Score standardization, etc., which will not be elaborated here; c1, c2, and c3 are set according to the actual situation. For example, the expert weighting method is adopted, that is, experts in related fields are invited to determine the preset proportionality coefficients of each index through professional opinion surveys and comprehensive evaluations. For example, c1, c2, and c3 can be 0.3, 0.4, and 0.3;

[0060] Compare the automatic downweighting delay coefficient with the preset automatic downweighting delay coefficient threshold. If the automatic downweighting delay coefficient is greater than the automatic downweighting delay coefficient threshold, it indicates that there is an obvious downweighting delay in the low-confidence mode, generate a downweighting delay signal, and mark the downweighting time period of this mode as the downweighting delay time period;

[0061] The resource anomaly analysis module is used to obtain the information of abnormal increase in computing resources during the automatic downweighting delay period of the low-confidence mode data and obtain the coefficient of abnormal increase in computing resources, and analyze the degree of abnormal increase in computing resources of the low-confidence mode data;

[0062] In the present invention, the abnormal increase coefficient of computing resources is a quantitative index used to measure the abnormal increase degree of the system computing resource occupancy load caused by low-confidence modal data during the automatic downgrading delay period. This coefficient not only reflects the instantaneous impact of the current modal input on computing resources, but also reveals the resource contention pressure, task scheduling backlog, and thread blocking trend accumulated by the system due to lagged processing. A larger abnormal increase coefficient of computing resources indicates that there are significant increases in computing resource consumption, abnormal peaks in CPU or memory utilization, and even deterioration in the processing performance of other tasks and response time during the system's processing of low-confidence modal data with delayed downgrading. Conversely, a smaller abnormal increase coefficient of computing resources indicates that although the low-confidence modal data is in the delayed downgrading stage, the system can still process it in a relatively stable and balanced resource allocation mode without inducing cascading performance degradation at the system level. In a multi-modal interaction system, there are significant differences in the processing priorities and resource occupancy models between different modal data. Due to high recognition failure or conflict rates, low-confidence modal data continues to occupy system threads, caches, and semantic matching engine resources before the downgrading takes effect in a timely manner. This "resource retention" will trigger cascading failures such as service delay amplification and context semantic drift, ultimately leading to global performance degradation. Evaluating the risk of resource blocking cascading imbalance in an existing intelligent customer service interaction system based on this abnormal increase coefficient of computing resources has significant engineering and system optimization value. This coefficient can be used as a dynamic indicator for system resource management. Combined with the automatic downgrading delay coefficient, it serves as the core parameter of the resource blocking cascading imbalance risk assessment model, improving the modeling accuracy and response speed of the system for the "slow modal drift, delayed downgrading - computing impact - response blocking" cascading effect. The abnormal increase coefficient of computing resources not only has the function of anomaly detection, but also plays a key role in the performance evaluation, resource scheduling, and risk response of an intelligent customer service interaction system. It is a key support indicator for ensuring the steady-state interaction ability of the system, suppressing the trend of irrational modal resource consumption, and realizing elastic control of multi-modal services. Through the real-time monitoring and evaluation of this indicator, a more robust resource regulation mechanism for multi-modal complex interaction scenarios can be constructed, effectively improving the response efficiency and service stability of the intelligent customer service system.

[0063] The acquisition logic of the abnormal increase coefficient of computing resources is as follows:

[0064] Sample the resource items consumed by running tasks on low-confidence modal data within the automatic downgrading delay time interval [ts, te] (the sampling interval is δt, and there are a total of N sampling points). The resource items include the CPU resource utilization rate R1(t), the memory resource utilization rate R2(t), the I / O resource utilization rate R3(t), and the thread resource utilization rate R4(t), and construct a resource matrix:

[0065] Compare the stable baseline time period [ts - T0, ts] before automatic weight reduction and calculate the relative perturbation amplitude: Where ΔR h (t) is the relative perturbation amplitude of the h-th resource, h = {1, 2, 3, 4}, R h (t) is the specific sampling value of the h-th resource, is the average utilization rate of the h-th resource within the stable baseline time period before automatic weight reduction; calculate the sliding change rate of each resource within the sliding window CK: Where VH h is the sliding change rate of the h-th resource, ΔR h (t - k) is the relative perturbation amplitude of the h-th resource calculated at time (t - k), ΔR h (t - k - 1) is the relative perturbation amplitude of the h-th resource calculated at time (t - k - 1);

[0066] Calculate the correlation coefficient between each pair of resources based on the dynamic covariance of the relative perturbation amplitude: Where Corr hg (t) is the correlation coefficient between the h-th resource and the g-th resource, ΔR h (τ) is the relative perturbation amplitude of the h-th resource calculated at time τ, is the average value of the relative perturbation amplitude of the h-th resource calculated within the sliding window CK, ΔR g (τ) is the relative perturbation amplitude of the g-th resource calculated at time τ, is the average value of the relative perturbation amplitude of the g-th resource calculated within the sliding window CK;

[0067] Calculate the coupling degree between each pair of resources: Where oh hg is the coupling degree between the h-th resource and the g-th resource, VH g is the sliding change rate of the g-th resource;

[0068] Calculate the resource fluctuation perception value of the h-th resource: Where gz h (t) is the resource fluctuation perception value of the h-th resource, ΔR g (t) is the relative perturbation amplitude of the g-th resource calculated at time t;

[0069] The calculation expression for calculating the resource abnormal rise coefficient is as follows: Where jsz is the calculated resource abnormal rise coefficient;

[0070] The resource imbalance risk assessment module is used to construct a resource blocking chain imbalance risk assessment model based on the automatic downgrading delay coefficient and the abnormal increase coefficient of computing resources, output the resource blocking chain imbalance risk index, and evaluate the resource blocking chain imbalance risk of the existing intelligent customer service interaction system;

[0071] Construct a resource blocking chain imbalance risk assessment model based on the automatic downgrading delay coefficient and the abnormal increase coefficient of computing resources, and output the resource blocking chain imbalance risk index. The formula on which the resource blocking chain imbalance risk assessment model is based is as follows: zyls = w1 * zdj + w2 * jsz, where zyls is the resource blocking chain imbalance risk index, zdj is the automatic downgrading delay coefficient, jsz is the abnormal increase coefficient of computing resources, w1 and w2 respectively represent the preset proportionality coefficients of the automatic downgrading delay coefficient and the abnormal increase coefficient of computing resources, and both w1 and w2 are greater than 0;

[0072] It should be noted that the above formulas are all calculated by taking their numerical values after removing the dimension. Common methods for removing dimensions include Min-Max normalization, Z-Score standardization, etc., which will not be elaborated here; w1 and w2 are set according to the actual situation. For example, the expert empowerment method is adopted, that is, experts in related fields are invited to determine the preset proportionality coefficients of each index through professional opinion surveys and comprehensive evaluations. For example, w1 and w2 can be 0.5 and 0.5;

[0073] It can be seen from the above calculation expressions that the larger the automatic downgrading delay coefficient and the larger the abnormal increase coefficient of computing resources, the larger the resource blocking chain imbalance risk index, indicating that in the current intelligent customer service interaction system, due to the coupling phenomenon of response lag and abnormal computing resource consumption in the processing of low-confidence modal data, a potential trend of systemic risk accumulation has been triggered, and resources are more likely to be blocked, delayed or even cascaded to failure in modal scheduling or task response. On the contrary, the smaller the automatic downgrading delay coefficient and the smaller the abnormal increase coefficient of computing resources, the smaller the resource blocking chain imbalance risk index, indicating that the current intelligent customer service interaction system has high response sensitivity and resource management efficiency in processing low-confidence modal data. The system can timely start the downgrading mechanism in the early stage when the input reliability of the modal decreases, effectively strip or weaken the interference of abnormal modes on resource allocation, and maintain the stable distribution of computing resources and control overhead during the automatic downgrading process, thus avoiding problems such as resource scheduling congestion, interaction blockage or service response delay caused by abnormal modal delay processing or multi-threaded task accumulation;

[0074] The risk response regulation module is used to compare the resource blocking chain imbalance risk index with the preset resource blocking chain imbalance risk index threshold, and conduct early warning grading on the resource blocking chain imbalance risk of the existing intelligent customer service interaction system, specifically as follows:

[0075] If the resource blocking chain imbalance risk index is greater than the resource blocking chain imbalance risk index threshold, it indicates that the current system is operating under high resource pressure, there are significant potential risks of resource imbalance, and a red warning signal is generated. When the red warning signal is generated, a one - key call mechanism is activated. The one - key call mechanism establishes a stable audio - video link based on the SIP call protocol. By connecting the front - desk voice terminal and the remote self - service machine, it realizes reverse remote control and real - time call connection: The system identifies high - confidence commands such as "call the front desk" from the user through multi - modal perception; activates the one - key call instruction and transmits the call signal to the remote front desk or the on - duty end; the SIP protocol establishes a point - to - point communication connection to open the service channel between the user's room and the hotel back - end; the self - service terminal screen can be remotely activated for display, or the operation interface is linked to present the hotel service request entry; the front - desk staff can remotely retrieve the current system operation status of the guest room through the remote interface, including modal recognition, resource scheduling, etc., to assist in judging whether manual takeover or restart of the modal service is required.

[0076] It should be noted that the needs of guests in any guest room in the hotel can be connected to the intelligent service center headquarters through voice commands via SIP calls to achieve one - to - many services. Similarly, guests can also use the one - key call button on the front - desk self - check - in machine in the hotel to quickly access the intelligent service center.

[0077] If the resource blocking chain imbalance risk index is less than or equal to the resource blocking chain imbalance risk index threshold, it indicates that the current system is in the safe range of resource scheduling, the resource allocation and modal interaction are operating well together, there are no obvious risk hazards, and a green warning signal is generated. When the green warning signal is generated, the current resource allocation strategy is maintained.

[0078] The present invention uses a multimodal perception and annotation module to synchronously collect multimodal signals and assign confidence tags, thereby achieving accurate identification and advance downgrade control of low-confidence modal data, and cutting off the interference source of redundant modalities on the main task path in advance; the downgrade delay analysis module dynamically quantifies the modal intervention risk caused by the automatic downgrade lag, and constructs the automatic downgrade delay coefficient as the core characterization indicator of real-time evaluation within the system; combined with the computing resource abnormality increase coefficient introduced in the resource anomaly analysis module, the actual impact of low-confidence modalities on computing resources during the downgrade lag window period can be accurately evaluated, thereby comprehensively quantifying the modality-resource conflict dynamics; further, the above two core indicators are integrated to construct A resource blocking chain imbalance risk assessment model is built to output a resource blocking chain imbalance risk index that represents the health of system operation, thereby realizing real-time assessment of potential blocking risks in the system. Finally, the risk response and control module compares the resource blocking chain imbalance risk index with the preset resource blocking chain imbalance risk index threshold, triggers graded warnings, and realizes efficient frequency reduction and flow limiting, task stripping, and main mode protection, ensuring that the high-confidence main mode obtains a stable computing path and makes a feedforward response to potential congestion risks, which not only greatly reduces the risks of resource conflicts and semantic deviations during multimodal fusion, but also significantly shortens the system response delay, thereby improving the accuracy of multimodal interaction and user satisfaction.

[0079] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0080] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0081] It should be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the above processes do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0082] If the described functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs, etc., which can store program codes.

[0083] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims described above.

Claims

1. A hotel guest room multimodal intelligent customer service interaction system based on a voice gateway, characterized in that: It includes a multi-modal perception annotation module, a down-weighting delay analysis module, a resource anomaly analysis module, a resource imbalance risk assessment module, and a risk response regulation module; The multi-modal perception annotation module is used to collect the modal signals of users through multi-source perception terminals deployed in the guest rooms to form a multi-modal perception input set, mark the confidence of each modality, and perform down-weighting scheduling on the modalities with low marked confidence; The down-weighting delay analysis module is used to obtain the automatic down-weighting delay information of the low-confidence modality data and obtain the automatic down-weighting delay coefficient, and analyze the degree of automatic down-weighting delay of the low-confidence modality data; The resource anomaly analysis module is used to obtain the information of abnormal increase in computing resources during the automatic down-weighting delay period of the low-confidence modality data and obtain the coefficient of abnormal increase in computing resources, and analyze the degree of abnormal increase in computing resources of the low-confidence modality data; The resource imbalance risk assessment module is used to construct a resource blocking chain imbalance risk assessment model based on the automatic down-weighting delay coefficient and the coefficient of abnormal increase in computing resources, output the resource blocking chain imbalance risk index, and evaluate the resource blocking chain imbalance risk of the existing intelligent customer service interaction system; The risk response regulation module is used to compare the resource blocking chain imbalance risk index with the preset resource blocking chain imbalance risk index threshold, and perform early warning grading on the resource blocking chain imbalance risk of the existing intelligent customer service interaction system.

2. The hotel guest room multi-modal intelligent customer service interaction system based on a voice gateway according to claim 1, wherein: The acquisition logic of the automatic down-weighting delay coefficient is as follows: Construct a modal response path graph Gl=(V, E) for multiple intermediate paths of fundamental modal processing, where V is the set of each sub-node of modal processing, and E is the set of edges, representing the operation jump between nodes. Each edge by mn corresponds to a modal processing jump, and obtain its processing time consumption chs from the system historical log mn ; Calculate the total response time of modal processing: zxt = ∑ (m,n)∈LJ chs mn , where zxt is the total response time of modal processing, and LJ is the actual path that the current mode experiences in the modal response path diagram; Calculate the modal time series drift entropy: Hshift = -∑ (m,n)∈LJ gl mn * log(gl mn ), where Hshift is the modal time series drift entropy, and gl mn is the time series distribution probability of the processing time between node m and node n, Define the weight perturbation response function \(q_{zr}(t)\) to characterize the dynamic behavior of the modal weight change during the weight reduction process. If the system completes the adjustment of the modal weight from \(t\) m to \(t\) n , then the weight perturbation response function \(q_{zr}(t)\) is expressed as where \(q_{z}\) init is the modal weight before adjustment, \(q_{z}(t)\) is the modal weight at time \(t\), and \(t\in[t\) m , \(t\) n ; Calculate the weight perturbation response value: where \(R_d\) is the weight perturbation response value. The calculation expression of the automatic down-weighting delay coefficient is as follows: zdj = c1*zxt + c2*Hshift + c3*Rd, where zdj is the automatic down-weighting delay coefficient, c1, c2, and c3 are the preset proportionality coefficients of the total modal processing response time, modal timing drift entropy, and weight perturbation response value respectively, and c1, c2, and c3 are all greater than 0.

3. The hotel guest room multimodal intelligent customer service interaction system based on a voice gateway according to claim 2, wherein: Compare the automatic down-weighting delay coefficient with the preset automatic down-weighting delay coefficient threshold. If the automatic down-weighting delay coefficient is greater than the automatic down-weighting delay coefficient threshold, generate a down-weighting delay signal and mark the down-weighting time period of this modality as the down-weighting delay time period.

4. The hotel guest room multi-modal intelligent customer service interaction system based on a voice gateway according to claim 1, characterized in that: The acquisition logic of the coefficient of abnormal increase in computing resources is as follows: Sample the resource items consumed by running tasks on the low-confidence modality data within the automatic down-weighting delay time interval [ts, te]. The resource items include the CPU resource utilization rate R1(t), the memory resource utilization rate R2(t), the I / O resource utilization rate R3(t), and the thread resource utilization rate R4(t), and construct a resource matrix; Compare the stable baseline time period [ts - T0, ts] before automatic downgrading, and calculate the relative perturbation amplitude: where ΔR h (t) is the relative perturbation amplitude of the h-th resource, h = {1, 2, 3, 4}, R h (t) is the specific sampling value of the h-th resource, is the average utilization rate of the h-th resource within the stable baseline time period before automatic downgrading; Calculate the sliding change rate of each resource within the sliding window CK: where VH h is the sliding change rate of the h-th resource, ΔR h (t - k) is the relative perturbation amplitude of the h-th resource calculated at time (t - k), ΔR h (t - k - 1) is the relative perturbation amplitude of the h-th resource calculated at time (t - k - 1); Calculate the correlation coefficient between each pair of resources based on the dynamic covariance of the relative perturbation amplitude: where Corr hg (t) is the correlation coefficient between the h-th resource and the g-th resource, ΔR h (τ) is the relative perturbation amplitude calculated for the h-th resource at time τ, is the mean value of the relative perturbation amplitude calculated for the h-th resource within the sliding window CK, ΔR g (τ) is the relative perturbation amplitude calculated for the g-th resource at time τ, is the mean value of the relative perturbation amplitude calculated for the g-th resource within the sliding window CK; Calculate the coupling degree between each pair of resources: where oh hg is the coupling degree between the h-th resource and the g-th resource, and VH g is the sliding change rate of the g-th resource; Calculate the resource fluctuation perception value of the h-th resource: where gz h (t) is the resource fluctuation perception value of the h-th resource, and ΔR g (t) is the relative perturbation amplitude calculated for the g-th resource at time t; The calculation expression for the abnormal increase coefficient of computing resources is as follows: where jsz is the abnormal increase coefficient of computing resources.

5. The hotel guest room multi-modal intelligent customer service interaction system based on a voice gateway according to claim 1, characterized in that: Construct a resource blocking chain imbalance risk assessment model based on the automatic down-weighting delay coefficient and the coefficient of abnormal increase in computing resources, output the resource blocking chain imbalance risk index. The formula on which the resource blocking chain imbalance risk assessment model is based is as follows: zyls = w1*zdj + w2*jsz, where zyls is the resource blocking chain imbalance risk index, zdj is the automatic down-weighting delay coefficient, jsz is the coefficient of abnormal increase in computing resources, w1 and w2 are the preset proportionality coefficients of the automatic down-weighting delay coefficient and the coefficient of abnormal increase in computing resources respectively, and w1 and w2 are all greater than 0.

6. The hotel guest room multimodal intelligent customer service interaction system based on a voice gateway according to claim 5, wherein: Compare the resource blocking chain imbalance risk index with the preset resource blocking chain imbalance risk index threshold, and conduct early warning grading on the resource blocking chain imbalance risk of the existing intelligent customer service interaction system as follows: If the resource blocking chain imbalance risk index is greater than the resource blocking chain imbalance risk index threshold, generate a red early warning signal and activate the one-key call mechanism; If the resource blocking chain imbalance risk index is less than or equal to the resource blocking chain imbalance risk index threshold, generate a green early warning signal.

Images