A real-time call assistance analysis method and system based on asr

Abstract

The application provides a real-time call auxiliary analysis method and system based on ASR, and relates to the technical field of voice data processing.The method comprises the following steps: obtaining a sequence of speech segments cut by a voice recognition engine according to timestamps, calculating the timestamp difference at the junction of adjacent speech segments to obtain a sequence of pause intervals; calculating the sentence-by-sentence change amount of the pause interval between the current speech segment and the previous speech segment according to the sequence of pause intervals to obtain a rhythm fluctuation sequence; extracting the fluctuation pattern of the three sentence-by-sentence change amounts arranged continuously along the time axis in the rhythm fluctuation sequence to construct a local pause fluctuation trend; dividing the segment with the first-increasing and then-decreasing convex shape of the local pause fluctuation trend into the rhythm imbalance area, and the segment with the gentle fluctuation into the normal expression area to obtain a labeled segment set.The application realizes the real-time auxiliary analysis process from analysis to intervention through the cross modulation and time dissipation accumulation of rhythm abnormalities on semantic weight.

Description

Technical Field

[0001] This invention relates to the field of voice data processing technology, and in particular to a real-time call assistance analysis method and system based on ASR. Background Technology

[0002] In real-time calls such as call centers, customer service, and remote negotiations, automatic speech recognition technology typically segments continuous speech streams into timestamped segments and outputs transcribed text. Existing real-time assistance solutions mostly generate prompts based on the average speech rate of the entire call, the silence duration of a single segment, the presence of keywords, or the emotional result of a single sentence. These solutions can provide basic speech transcription and content retrieval, but their analysis objects are usually limited to global acoustic statistics or isolated text fragments, and they do not preserve the sentence-by-sentence changes in pause intervals according to the continuous temporal relationship between adjacent segments.

[0003] However, when the pauses in the response voice are first extended and then shortened and occur continuously with sensitive semantics, the existing solution cannot identify the fluctuations of the pauses on the same call timeline, use the amplitude of rhythm imbalance to modulate the semantic weight of the corresponding word, and generate agent intervention information that is adapted to the current state in real time based on the cumulative results. Therefore, the prompts obtained by the agent terminal are likely to not correspond to the actual time and degree of rhythm-semantic change. Summary of the Invention

[0004] This invention provides a real-time call auxiliary analysis method and system based on ASR, which realizes a real-time auxiliary analysis process from analysis to intervention by cross-modulating semantic weights and accumulating time dissipation through rhythm anomalies.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: Firstly, a real-time call-assisted analysis method based on ASR, the method comprising: Obtain the speech segment sequence segmented by timestamp by the speech recognition engine, calculate the timestamp difference at the boundary of adjacent speech segments, and obtain the pause interval sequence; Based on the pause interval sequence, calculate the sentence-by-sentence change in the pause interval between the current segment and the previous segment to obtain the rhythm fluctuation sequence; extract the fluctuation pattern of the three sentence-by-sentence changes that are continuously arranged along the time axis in the rhythm fluctuation sequence to construct the local pause fluctuation trend. Segments with local pauses and fluctuations that show an initial increase followed by a decrease and a convex shape are classified into the rhythm imbalance zone, while those with smooth fluctuations are classified into the normal expression zone, resulting in a set of labeled segments. The corresponding response speech segments are extracted from the rhythm imbalance area in the labeled segment set and transcribed to obtain the imbalanced response text segment; the word units in the imbalanced response text segment are extracted and matched with the preset high-pressure sensitive word library, and the semantic negative weight is assigned according to the product of the preset sensitivity intensity and the rhythm imbalance amplitude to construct the semantic deviation sequence; the accumulation range is defined segment by segment in the rhythm imbalance area, and the semantic negative weight in the semantic deviation sequence is accumulated sentence by sentence to obtain the rhythm-semantic pressure index; Determine whether the rhythm-semantic pressure index exceeds the preset intervention trigger threshold, and trigger the generation of a speech downgrade prompt signal; match the corresponding downgrade strategy label according to the pressure level range into which the rhythm-semantic pressure index falls; The downgrade prompt signal and downgrade strategy label are encapsulated into auxiliary intervention instructions to generate an intervention message package; the auxiliary intervention instructions are sent to the agent terminal interface, where visual script guidance information is displayed and real-time call auxiliary analysis results are output.

[0006] Secondly, the real-time call assistance analysis system based on ASR includes: The acquisition module is used to acquire the speech segment sequence segmented by the speech recognition engine according to the timestamp, calculate the timestamp difference at the boundary of adjacent speech segments, and obtain the pause interval sequence. The extraction module is used to calculate the sentence-by-sentence change of the pause interval between the current segment and the previous segment based on the pause interval sequence, and obtain the rhythm fluctuation sequence; extract the fluctuation pattern of the three sentence-by-sentence changes arranged continuously along the time axis in the rhythm fluctuation sequence to construct the local pause fluctuation trend. The "Classification" module is used to classify segments with local pauses and fluctuations that show an initial increase followed by a decrease in a convex shape into the rhythm imbalance zone, and segments with smooth fluctuations into the normal expression zone, thus obtaining a set of labeled segments. The transcription module is used to extract the corresponding response speech segments from the rhythm imbalance area in the set of labeled segments and transcribe them to obtain the imbalanced response text segments; extract the word units in the imbalanced response text segments and match them with the preset high-pressure sensitive word library, assign semantic negative weights according to the product of preset sensitivity intensity and rhythm imbalance amplitude, and construct semantic deviation sequences. The calculation module is used to define the cumulative range segment by segment within the rhythm imbalance zone, and accumulate the semantic negative weights in the semantic deviation sequence sentence by sentence to obtain the rhythm-semantic pressure index; determine whether the rhythm-semantic pressure index exceeds the preset intervention trigger threshold, trigger the generation of the speech downgrade prompt signal; and match the corresponding downgrade strategy label according to the pressure level range into which the rhythm-semantic pressure index falls. The control module is used to encapsulate the script degradation prompt signal and degradation strategy label into auxiliary intervention instructions, generate intervention message packages, and send the auxiliary intervention instructions to the agent terminal interface. The agent terminal interface displays visual script guidance information and outputs real-time call auxiliary analysis results.

[0007] The above-described solution of the present invention has at least the following beneficial effects: The speech segment sequence is obtained by the speech recognition engine according to the timestamp. The pause interval sequence is formed by the timestamp difference at the boundary of adjacent speech segments. This can locate the rhythm imbalance area corresponding to local pause fluctuations during the call and provide clear speech segment boundaries and time anchors for subsequent semantic processing.

[0008] Based on the rhythm imbalance region, the corresponding response speech segment is extracted and the imbalanced response text segment is obtained. The text word units are matched with the preset high-pressure sensitive word library, and then the semantic negative weight is formed by multiplying the preset sensitivity intensity and the rhythm imbalance amplitude. Within the same rhythm imbalance region, the weight is reduced according to the time distance and accumulated sentence by sentence to obtain the rhythm-semantic pressure index. Since the weight of the same word unit is modulated by the local rhythm state, and word units at different distances from the imbalance start boundary participate in the accumulation according to the determined dissipation relationship, it is possible to convert the local acoustic fluctuations and corresponding semantic content into a unified control quantity with continuous time meaning, avoiding the generation of the same intensity prompt based solely on whether the keyword appears or the single sentence text result.

[0009] The rhythm-semantic stress index is compared with the intervention trigger threshold and the stress level range. The downgrade strategy label is matched according to the index range. The downgrade prompt signal and the downgrade strategy label are then encapsulated into an auxiliary intervention instruction and sent to the agent terminal interface. Since the prompt trigger time, strategy level and display content are all derived from the aforementioned local rhythm-semantic processing results, the agent terminal can obtain executable speech guidance information in the corresponding call stage. The subsequent voice stream enters a new analysis cycle, overcoming the shortcomings of existing solutions that cannot jointly process local pauses and fluctuations with corresponding semantics on the same time axis and generate adaptive agent intervention information in real time. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating a real-time call assistance analysis method based on ASR provided by an embodiment of the present invention.

[0011] Figure 2 This is a schematic diagram of a real-time call assistance analysis system based on ASR provided by an embodiment of the present invention.

[0012] Figure 3 This is a flowchart illustrating how an embodiment of the present invention encapsulates a script degradation prompt signal and a degradation strategy label into an auxiliary intervention instruction, generates an intervention message package, sends the auxiliary intervention instruction to the agent terminal interface, displays visual script guidance information on the agent terminal interface, and outputs real-time call auxiliary analysis results. Detailed Implementation

[0013] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0014] The following is combined Figures 1 to 3 The embodiments of this application will be described in further detail. The described embodiments are used to illustrate this application and are not intended to limit the scope of protection of this application. In the absence of logical conflicts, the technical features in this embodiment can be combined with each other. The expressions such as "first", "second", etc. are only used to distinguish objects and do not indicate order or importance. ASR is the abbreviation for Automatic Speech Recognition.

[0015] In this embodiment, a speech segment is a continuous speech data unit formed by the speech recognition engine based on the speech activity boundary, silence boundary, or incremental transcription boundary. Each speech segment is associated with at least a speech segment identifier, a speaker or channel identifier, a start timestamp, an end timestamp, and optional transcribed text. The start timestamp and end timestamp use the same clock source and are converted to the same time unit. The timestamp can be an offset relative to the start time of the call or an absolute time after synchronization.

[0016] The pause interval referred to in this embodiment is the difference between the start timestamp of the next segment and the end timestamp of the previous segment between two adjacent segments of the same response subject. The sentence-by-sentence change is the change result of the next pause interval relative to the previous pause interval on the time axis. The local rhythm triplet is three consecutive sentence-by-sentence changes in the rhythm fluctuation sequence. All of the above data are generated from the call audio, ASR segmentation results and timestamps, and belong to the time structure data in the speech signal processing process.

[0017] The rhythm-semantic pressure index referred to in this embodiment is a call assistance control index obtained by coupling local pause changes with the matching results of a preset word library. Its purpose is to select whether to display guidance information to reduce the intensity of the speech to the agent terminal. This index is not used to diagnose the speaker's psychological illness, mental state, personality traits or true subjective intentions, nor is it used as the sole basis for automatic denial of service, automatic pricing, automatic credit granting, personnel assessment or other actions that have a substantial impact on personal rights.

[0018] This embodiment can be applied to legally authorized call centers, customer service, remote negotiation, or other real-time voice interaction scenarios. The system includes a call access service, an ASR speech recognition engine, an auxiliary analysis service, configuration storage, a message routing service, and agent terminals. The call access service receives bidirectional audio streams and extracts the response speech stream to be analyzed based on the channel identifier or speaker separation result. The ASR speech recognition engine outputs a time-stamped speech segment sequence. The auxiliary analysis service sequentially performs rhythm analysis, segment marking, semantic weighting, exponential accumulation, and strategy matching. The message routing service sends intervention message packets to the agent terminals bound to the current call identifier.

[0019] The configuration storage is used to save morphological tolerance, area threshold, sharpness threshold, intervention trigger threshold, oppression level range, high-pressure sensitive word library, word sensitivity intensity, basic dissipation ratio, step increment attenuation coefficient, and speech degradation strategy library. Each configuration item carries a version number, effective time, and applicable scenario identifier, which helps the analysis service to use the same configuration version in the calculation of a rhythm imbalance zone. When the configuration is updated in the middle of the process, it will take effect from the next rhythm imbalance zone or the next call session, thereby avoiding the drift of the judgment benchmark within the same accumulation range.

[0020] The call access service, ASR speech recognition engine, and auxiliary analysis service can be deployed on the same server, or they can be deployed on edge nodes and cloud servers respectively. In a low-latency implementation, the call access service and ASR speech recognition engine are located on edge nodes close to the agent terminal, the auxiliary analysis service completes timestamp and lexical processing on the edge node, the configuration storage provides the current configuration through a read-only cache, and the message routing service uses persistent connections to push intervention messages to the agent terminal, thereby shortening the data transmission path from the end of the call segment to the display of the guidance information.

[0021] The ASR speech recognition engine in this embodiment is a speech processing component that receives digital speech sampling sequences and outputs segment identifiers, segment start timestamps, segment end timestamps, transcribed text, and stable states. Its input can be a monophonic response speech stream in pulse code modulation format. The input record includes at least a call identifier, channel identifier, sampling rate, segment start sampling position, and speech sampling value. The timestamps of its output segments are all relative to the same call start time. The auxiliary analysis service only uses the ASR speech recognition engine that satisfies this input-output constraint.

[0022] In one implementation, the ASR speech recognition engine first divides the continuous speech sampling sequence into overlapping short-time speech frames. Each speech frame contains a fixed number of temporally continuous sampled values, and the starting sampling positions of adjacent speech frames differ by a fixed number of frame shift samples. The engine then sums the squares of each sampled value in each speech frame, and uses the ratio of the sum of squares to the number of samples in that speech frame as the short-time energy of that speech frame. The non-speech frame energy at the beginning of the call is sorted from smallest to largest, and the energy corresponding to the preset sorting position is taken as the current noise benchmark.

[0023] The ASR speech recognition engine compares the short-time energy of each speech frame with the sum of a noise baseline and a speech activity increment threshold. If the short-time energy is greater than the sum, the speech frame is marked as an active frame; otherwise, it is marked as an inactive frame. When the number of consecutive active frames reaches the initial acknowledgment count, the starting sampling position of the first frame in the consecutive active frames is used as the starting point of the candidate segment. When the number of consecutive inactive frames reaches the end acknowledgment count, the starting sampling position of the first frame in the consecutive inactive frames is used as the ending point of the candidate segment. Candidate segments shorter than the minimum segment length are merged into adjacent segments or marked as invalid segments.

[0024] For each speech frame in a candidate segment, the ASR speech recognition engine forms an acoustic feature sequence according to the temporal order of the speech frames. The acoustic features can be obtained directly from the normalized sample values ​​by the trained end-to-end acoustic encoder. During normalization, the average value of all sample values ​​in the candidate segment is first calculated. The average value is then subtracted from each sample value to obtain the deDC sample value. The average of the squares of each deDC sample value is then calculated and the square root is taken to obtain the amplitude scale. When the amplitude scale is greater than zero, the normalized sample value is obtained by the ratio of the deDC sample value to the amplitude scale. When the amplitude scale is equal to zero, the candidate segment is marked as having no valid speech.

[0025] The acoustic encoder can employ a neural network with temporal sequence modeling capabilities. The training data includes authorized speech samples and corresponding labeled text for each speech sample. During training, the encoder outputs the probabilities of candidate characters and whitespace markers for each time position. The training program enumerates the alignment paths of the labeled text by deleting whitespace markers and merging consecutive repeated characters. The probabilities corresponding to each time position in each alignment path are multiplied to obtain the path probability. The probabilities of all valid paths are then added together to obtain the labeled text probability. The negative logarithm of the labeled text probability is used as the recognition loss.

[0026] The training program determines the direction and degree of change of the recognition loss relative to each network parameter according to the backpropagation order. The product of the learning step size and the degree of change is used as the parameter adjustment amount, and the parameter adjustment amount is subtracted from the current parameters to obtain the updated parameters. After each round of parameter update, the training program uses independent verification speech to calculate the verification loss. Training stops when the verification loss does not decrease for a preset number of rounds, the number of training rounds reaches the upper limit, or the parameter change amount is less than the stopping threshold. The parameters corresponding to the rounds with lower verification loss are saved as acoustic encoder parameters.

[0027] During inference, the ASR speech recognition engine starts building a candidate text set from the first acoustic time position of the candidate segment. At each time position, it expands the current candidate text with the probability of each candidate character at that position. The product of the cumulative probability of the candidate text before expansion and the probability of the current character is calculated as the expanded candidate probability. Candidate texts with probabilities lower than the retention criteria are deleted. When the number of candidates exceeds the upper limit of the beam width, only the aforementioned upper limit number of candidate texts are retained in descending order of cumulative probability. After processing the last time position, blank markers are deleted and consecutive repeated characters are merged. The candidate text with the highest cumulative probability is determined as the transcribed text.

[0028] The ASR speech recognition engine finds the acoustic time position of the first and last occurrence of each character based on the character alignment path of the optimal candidate text. It multiplies the acoustic time position by the corresponding frame shift duration and adds the candidate segment start time to obtain the character time range. The segment start timestamp is taken from the start time of the first valid character in the candidate segment or the start point of the candidate segment, and the segment end timestamp is taken from the end time of the last valid character or the end point of the candidate segment. When no valid characters are formed, the acoustic segment boundary is preserved and the transcribed text is marked as empty.

[0029] When the ASR speech recognition engine adopts an external mature engine, as long as the external engine can provide the same segment identifier, start and end timestamps under the same time base, transcribed text and stable state as defined above, it can replace the above acoustic encoder. Since the subsequent pause interval, rhythm change and semantic matching are all coupled with the ASR speech recognition engine through explicit output fields, replacing the engine will not change the progressive processing relationship from the segment time boundary to the agent intervention information in this application.

[0030] Step S101, generating a pause interval sequence, including: Upon receiving the speech segment events output by the ASR speech recognition engine, the speech segments belonging to the same response speech stream are first grouped into the same buffer queue based on the call identifier and the respondent entity identifier, and then the speech segments are arranged in ascending order of start timestamp. When ASR uses incremental transcription and revises the same speech segment multiple times, the event with the newer version number is retained based on the speech segment identifier. When the version numbers of the two versions are the same, the event with the newer end timestamp is retained, thereby avoiding the same speech segment being calculated repeatedly.

[0031] For adjacent speech segments after sorting, the start timestamp of the next speech segment and the end timestamp of the previous speech segment are read. The time difference between the start time of the next speech segment and the end time of the previous speech segment is used as the silence duration. When the time difference is greater than zero, the time difference is recorded as the pause interval. When the time difference is equal to zero, the pause interval is recorded as zero. When the time difference is less than zero and the two speech segments belong to overlapping speech or ASR boundary correction, the pause interval is truncated to zero and the overlap identifier is recorded. When the time difference is less than zero and exceeds the allowable timestamp backoff tolerance, the calculation of the adjacent relationship is paused and the speech segment boundary is awaited for correction.

[0032] The effective silence duration is arranged along the continuous call time axis according to the corresponding boundary time to form a pause interval sequence. Each pause interval record includes the previous segment identifier, the next segment identifier, the previous segment end timestamp, the next segment start timestamp, silence duration, and quality identifier. When any adjacent segment is missing a start timestamp or end timestamp, it is not filled with the default value, but the boundary is marked as missing, and the window containing the missing position is skipped in the subsequent sliding window, thereby avoiding the formation of false protrusions by unfounded time values.

[0033] Because the pause interval sequence preserves the boundary time, silence duration and quality status between adjacent speech segments, subsequent processing can compare the changes of consecutive pauses on the same time reference, and can reverse locate the original response speech segment after the abnormal window is detected, thus establishing a traceable time anchor point for rhythm analysis and semantic transcription.

[0034] Step S102, generating a rhythmic fluctuation sequence, including: Processing begins from the second valid pause interval in the pause interval sequence. The current pause interval is compared with the immediately preceding pause interval on the time axis. The sentence-by-sentence change is obtained by subtracting the preceding pause interval from the current pause interval. A sentence-by-sentence change greater than 0 indicates that the current pause is longer than the previous pause, while a sentence-by-sentence change less than 0 indicates that the current pause is shorter than the previous pause. A sentence-by-sentence change equal to 0 indicates that the two consecutive pauses have the same duration.

[0035] Each sentence-by-sentence change is associated with the boundary time of the current pause interval, the two pause interval identifiers that constitute the change, and the corresponding speech segment identifier. The sentence-by-sentence changes are arranged according to the boundary time to form a rhythmic fluctuation sequence. If there is a missing position between two pause intervals, crosses the speaker switching position, or crosses the preset long interruption boundary, the two are not regarded as continuous pause intervals on the time axis, and the change sequence is re-established from the first complete position after the interruption boundary.

[0036] Since the sentence-by-sentence variation describes the relative change between consecutive pauses, rather than the average speech rate of the entire conversation, it can preserve the time pattern of local pauses that are first extended and then shortened or first shortened and then extended. At the same time, cross-subject, missing timestamps and long interruptions are not included in the continuous comparison, thereby reducing the interference of different expression processes on the local rhythm trend.

[0037] Step S103, constructing a local pause and fluctuation trend, including: A sliding window with a length of three changes and a step size of one change is set on the rhythm fluctuation sequence. Each time, the first, second, and third sentence changes arranged continuously along the time axis are read to form a local rhythm triplet. Each time the window moves, the two changes that overlap with the previous window are retained, thereby making adjacent local trends continuous on the time axis and accurately recording the position where the pattern first changes.

[0038] Using the boundary moments of the three sentence-by-sentence changes as the horizontal time positions and the corresponding sentence-by-sentence changes as the vertical positions, a first change node, a second change node, and a third change node are generated in the two-dimensional plane of time-rhythm change. Then, the first change node is connected to the second change node, and the second change node is connected to the third change node in sequence to obtain a local rhythmic polyline segment composed of two line segments. When the boundary moments use different time units, they are first uniformly converted to seconds or milliseconds. When there are identical or reversed moments among the three boundary moments, the window is marked as an invalid window of the time axis and no morphological judgment is performed.

[0039] In an equidistant implementation, the horizontal positions of the three nodes can be set as consecutive first, second, and third positions, respectively, with only the vertical sentence-by-sentence change retained for comparison. In this case, the unit of the local rhythm imbalance area is the time change multiplied by the number of position steps, and the denominator of the local rhythm sharpness is the two position step lengths. When using the actual boundary moment or equidistant positions, the threshold in the same deployment adopts the same calibration method as the selected horizontal coordinate, and the two are not mixed in the same configuration version.

[0040] Step S104, determine the local morphology and form a set of labeled fragments, including: First, determine the first and last line connecting the first and third variable nodes. Then, determine the shortest vertical distance from the second variable node to the first and last line and record this distance as the local rhythmic arch height. When the first and last nodes are in different lateral positions but in the same longitudinal position, the vertical distance is equal to the absolute deviation of the second variable relative to the horizontal line connecting the first and last variables. When the first and last line is inclined, the vertical distance is determined according to the shortest distance from the second node to the inclined line. When the first and last nodes coincide, the window is marked as a geometrically invalid window.

[0041] When calculating the local rhythmic arch height corresponding to the inclined first and last line, first subtract the lateral position of the first variable node from the lateral position of the third variable node to obtain the first and last lateral difference, and subtract the longitudinal position of the first variable node from the longitudinal position of the third variable node to obtain the first and last longitudinal difference; then subtract the lateral position of the first variable node from the lateral position of the second variable node to obtain the middle lateral difference, and subtract the longitudinal position of the first variable node from the longitudinal position of the second variable node to obtain the middle longitudinal difference.

[0042] Multiply the first and last horizontal difference with the middle vertical difference to obtain the first cross result, and multiply the first and last vertical difference with the middle horizontal difference to obtain the second cross result. Take the absolute value of the difference between the first and second cross results as the area-type intermediate quantity. Then, square the first and last horizontal difference and the first and last vertical difference respectively, add the two squared results and take the square root of the sum to obtain the length of the first and last line. Use the ratio of the area-type intermediate quantity to the length of the first and last line as the vertical deviation distance from the second variable node to the first and last line.

[0043] When the length of the first and last line is zero, the above ratio calculation is not performed and the window is marked as a geometrically invalid window; when using equidistant horizontal positions, the first and last horizontal difference is two position steps; when using actual timestamps, the first and last horizontal difference is the time span between the third node time and the first node time. The unit of the obtained local rhythmic arch height is consistent with the unit of sentence-by-sentence change and is directly used for morphological tolerance comparison and sharpness calculation.

[0044] Using the time reference axis with zero vertical change as the area reference, the areas enclosed by the local rhythmic broken line segments between the first and second nodes and between the second and third nodes and the time reference axis are calculated respectively. The two areas are then added together to obtain the area of ​​local rhythmic imbalance. When any line segment crosses the time reference axis, the line segment is divided at the intersection point, and the areas on both sides are calculated separately and then the absolute values ​​are added together to avoid the cancellation of positive and negative areas. When the actual time coordinate is used, the area dimension is the product of the unit of change of each sentence and the time unit.

[0045] When calculating the area between adjacent nodes on the same side and the time reference axis, the absolute values ​​of the vertical positions of the two nodes are obtained respectively. The two absolute values ​​are added together and divided by two to obtain the average height. Then, the horizontal position of the next node is subtracted from the horizontal position of the previous node to obtain the horizontal span. The average height is multiplied by the horizontal span to obtain the area of ​​the trapezoid corresponding to the adjacent line segment. When either of the two nodes is located on the time reference axis, its absolute height is taken as zero and included in the above average height calculation.

[0046] When calculating the area of ​​adjacent line segments crossing the time reference axis, first obtain the absolute values ​​of the vertical positions of the preceding and following nodes respectively. Use the ratio of the absolute value of the preceding node to the sum of the two absolute values ​​as the relative proportion of the intersection point. Then multiply the horizontal span by the relative proportion of the intersection point and add it to the horizontal position of the preceding node to obtain the intersection point of the broken line segment and the time reference axis. Subsequently, calculate the area of ​​the first triangle using the absolute height of the preceding node and the span from the preceding node to the intersection point. Calculate the area of ​​the second triangle using the absolute height of the following node and the span from the intersection point to the following node. Add the areas of the two triangles to obtain the area of ​​the adjacent line segment.

[0047] When the vertical positions of two adjacent nodes are both zero, the area of ​​the adjacent line segment is 0. When the sum of the two absolute values ​​is 0, the division of the relative proportion of the intersection point is not performed. The area from the first node to the second node is added to the area from the second node to the third node to obtain the local rhythm imbalance area. This area is saved together with the corresponding window identifier and area calculation method for use in area threshold comparison and recalculation of affected windows.

[0048] The span distance between the first and third variable nodes along the time axis is extracted, and the local rhythm sharpness is represented by the ratio of the local rhythm arch height to the span distance. When the span distance is less than the minimum effective span, the ratio calculation is not performed and the window is marked as invalid. When the span distance is valid, the local rhythm sharpness increases with the increase of the arch height and decreases with the increase of the span. Therefore, it is possible to distinguish local fluctuations with different durations under the same deviation amplitude.

[0049] When calculating the local rhythm sharpness, the first and last spans are obtained by subtracting the first change node's horizontal position from the third change node's horizontal position. The first and last spans are compared with 0 and the minimum effective span. If the first and last spans are not greater than zero or less than the minimum effective span, an invalid sharpness state is output. If the first and last spans are valid, the ratio of the local rhythm arch height to the first and last spans is used as the local rhythm sharpness. When using actual timestamps, the sharpness unit is the sentence-by-sentence change unit divided by the time unit. When using equally spaced positions, the sharpness unit is the sentence-by-sentence change unit divided by the position steps.

[0050] For a valid window, first determine whether the second sentence-by-sentence change is greater than the first sentence-by-sentence change and the third sentence-by-sentence change, to confirm that the line segment changes from rising to falling in time sequence. Then determine whether the local rhythm arch height is greater than the preset shape tolerance, whether the local rhythm imbalance area is greater than the preset area threshold, and whether the local rhythm sharpness is greater than the preset sharpness threshold. When the above conditions are met simultaneously, the window is marked as a first-increase-then-decrease convex shape. When the local rhythm arch height is less than or equal to the preset shape tolerance, the window is marked as a smooth fluctuation shape. The remaining windows are marked as transition shapes and wait for the adjacent windows to provide continuous information.

[0051] Morphological tolerance, area threshold, and sharpness threshold can be calibrated offline using authorized historical call samples. During calibration, different sampling rates and time units are first uniformly converted, and the camber, area, and sharpness distributions of normal continuous expression windows and windows to be intervened are statistically analyzed. Then, a combination of thresholds that can distinguish between the two types of windows is selected from the candidate thresholds that meet the preset false trigger constraints. When there are not enough calibration samples, a conservative configuration can be used to increase the trigger threshold, and the results are manually reviewed for subsequent configuration updates. However, specific production values ​​are not fabricated in this embodiment.

[0052] When constructing the candidate set of morphological thresholds, the calibration program collects the local rhythmic arch height, local rhythmic imbalance area, and local rhythmic sharpness of all labeled windows. It sorts the feature values ​​of each type from smallest to largest and removes duplicate values. For two adjacent but unequal feature values ​​after sorting, the calibration program divides the sum of the two feature values ​​by two to obtain the candidate boundary value. At the same time, it adds the boundary below the minimum feature value and the boundary above the maximum feature value to the candidate set, thereby obtaining the candidate set of arch height, the candidate set of area, and the candidate set of sharpness, respectively.

[0053] The calibration program sequentially combines a candidate value for arch height, a candidate value for area, and a candidate value for sharpness. For each labeled window, it performs the same directional relationship and three-threshold joint judgment as the online service. The calibration program counts the number of normal description windows judged as convex shapes and the total number of normal description windows. The ratio of the former to the latter is used as the false trigger ratio of the candidate combination. The program also counts the number of windows to be intervened judged as convex shapes and the total number of windows to be intervened. The ratio of the former to the latter is used as the recognition ratio of the candidate combination.

[0054] The calibration procedure first deletes candidate combinations whose false trigger ratios do not meet the configuration constraints, and then selects combinations with higher recognition ratios from the remaining combinations. When multiple combinations have the same recognition ratio, the combination with a lower false trigger ratio is selected first. If the false trigger ratios are still the same, the combination with a larger arch height threshold, a larger area threshold, and a larger sharpness threshold is selected in sequence to form a definite selection order. Finally, the three candidate values ​​in the combination are used as the morphological tolerance, area threshold, and sharpness threshold, respectively.

[0055] When the remaining candidate combinations are empty, the calibration program does not generate a new threshold version, but outputs a calibration failure status and continues to use the previous effective version; when the normal expression window set or the window set to be intervened is empty, the denominator of the corresponding proportion is zero, the calibration program stops the proportion calculation and outputs the sample category missing status, the deployment personnel supplement the samples and re-execute the calibration, and does not replace the missing category with the default proportion.

[0056] When constructing the amplitude reference upper limit, the calibration program sorts the original rhythm imbalance amplitudes of the confirmed convex window from smallest to largest, and reads the amplitude corresponding to the sorting position specified in the configuration as a candidate reference upper limit. This candidate reference upper limit must be greater than zero and not less than the minimum effective amplitude used to form the convex window. If the condition is met, a new configuration version with the same morphological threshold is written. If the condition is not met, the previous reference upper limit is retained, so that the amplitude mapping range is consistent with the version of the current morphological judgment sample.

[0057] Arrange the increasing-decreasing raised markers, smooth fluctuation markers, and transition markers according to the end time of the window to obtain a local morphological marker sequence; when an adjacent window switches from a smooth fluctuation marker to an increasing-decreasing raised marker, the start time of the next window is recorded as the morphological jump time; when multiple consecutive windows are all increasing-decreasing raised patterns, their time ranges are merged into a rhythm imbalance zone; when there are only no more than a preset number of transition windows between two raised windows, the transition windows can be merged into the same rhythm imbalance zone; when the number of transition windows exceeds this number or a smooth fluctuation marker appears, the current rhythm imbalance zone ends.

[0058] For each rhythm imbalance zone, the corresponding speech segment is searched in reverse based on the pause interval identifier referenced by each local rhythm triplet in the zone. The continuous response speech segments that intersect with the interval time range are added to the abnormal pause segment set and labeled as rhythm imbalance zone. For the speech segments corresponding to the continuous smooth fluctuation window, they are added to the smooth pause segment set and labeled as normal expression zone. The transition form is only used for regional connection judgment and does not trigger intervention independently. The labeled segment set is composed of the abnormal pause segment set and the smooth pause segment set.

[0059] Since the convex shape determination simultaneously constrains the directional relationship between the rise and fall, the arch height, the area, and the sharpness, a single timestamp jitter will only enter the rhythm imbalance zone when it forms a complete local shape and exceeds the corresponding threshold. At the same time, the local geometric results are mapped back to the continuous response segment through the shape jump moment and window merging rules, thereby defining a clear audio range for subsequent semantic analysis.

[0060] Step S105, construct the semantic deviation sequence and calculate the rhythm-semantic pressure index, including: For each rhythm imbalance region in the tagged segment set, the response speech segment is extracted from the call audio buffer according to the start and end timestamps of the region, and a context buffer not exceeding the boundary of the adjacent speech segment is retained forward and backward respectively. The stable transcribed text already output by the ASR speech recognition engine can be reused, or the extracted response speech segment can be re-recognized by calling the high-precision transcription channel. When the secondary transcription result carries word-level timestamps, word-level timestamps are used first. When only sentence-level timestamps are carried, the words in the sentence are associated with the corresponding sentence-level time range to obtain the imbalanced response text segment corresponding to the rhythm imbalance region on the time axis.

[0061] The imbalanced response text segment is subjected to character normalization, case unification, full-width / half-width conversion, and word segmentation to obtain an imbalanced response word set arranged in chronological order. For words with word-level timestamps, they are directly sorted according to the start and end timestamps. For words with only sentence-level timestamps, they are first sorted by sentence-level timestamps and then the original order within the sentence is maintained. When the ASR output is empty text, the confidence level is lower than the available threshold, or it only contains unresolved symbols, the sentence is marked as having no valid words, its rhythm data is retained, but no negative semantic weight is generated.

[0062] Each entry in the high-pressure sensitive word library includes at least a standardized word or phrase, a preset sensitivity intensity, an applicable language, a version number, and an optional business scenario identifier. The preset sensitivity intensity is a dimensionless configuration value. During matching, continuous multi-word phrase matching is performed first, followed by single-word word matching. When multiple candidate entries cover the same word range, the entry with the longer coverage length is retained. If the coverage lengths are the same, the entry with the higher preset sensitivity intensity is retained. If they are completely identical, the entry with the earlier word order in the word library is retained. This avoids the same word being assigned weight repeatedly due to overlapping entries.

[0063] For each convex window that first increases and then decreases within the rhythm imbalance zone, the absolute difference between the intermediate change and the first change, and the absolute difference between the intermediate change and the third change are determined respectively. The average of the two absolute differences is then used as the original rhythm imbalance amplitude of the window. When the same segment is covered by multiple convex windows, the maximum value among the original rhythm imbalance amplitudes covering the segment is used as the original rhythm imbalance amplitude of the segment, so as to preserve more obvious local fluctuations in the overlapping windows.

[0064] To enable the rhythm amplitude to be combined with the dimensionless preset sensitivity intensity, the original rhythm imbalance amplitude can be mapped to a preset value range of 0 to 1 based on the amplitude reference upper limit in the current configuration version, thus obtaining the rhythm imbalance amplitude used for weighting. When the original rhythm imbalance amplitude is not greater than zero, it is mapped to zero; when it is within the reference range, the relative size order is maintained; when it exceeds the reference upper limit, it is mapped to 1. The reference upper limit is determined by calibration samples using the same time unit as the morphological threshold. When the reference upper limit is missing or not greater than 0, semantic weighting is stopped, an abnormal configuration status is output, and no division by zero operation is performed.

[0065] The words or phrases in the imbalanced response word set are matched with the high-pressure sensitive word library. For successfully matched entries, their preset sensitivity intensity is multiplied by the rhythm imbalance amplitude corresponding to the segment in which the word is located to obtain the semantic negative weight of the word or phrase. For unmatched entries, the semantic negative weight is recorded as zero. When a matched phrase spans two segments, the rhythm imbalance amplitude corresponding to the segment with a larger overlap ratio with the phrase duration is used. If the overlap ratios are the same, the amplitude corresponding to the later segment is used. The weighted results are arranged in the word time order to form a semantic deviation sequence.

[0066] A segment-by-segment accumulation range is defined by the starting and ending boundaries of each rhythm imbalance zone, and sentences or stable segments are used as accumulation units. For the current word to be processed, the sequence number of the accumulation unit where the word is located relative to the starting unit of the rhythm imbalance zone is determined, and the number of accumulation units traversed between the two is taken as the time distance step. Words located in the starting unit correspond to zero time distance steps, and then one time distance step is added for each accumulation unit crossed thereafter.

[0067] The distance reduction parameter is determined based on the basic dissipation ratio, the number of time distance steps, and the step-incremental decay coefficient. Specifically, the product of the number of time distance steps and the step-incremental decay coefficient is first used as the incremental dissipation share. Then, the incremental dissipation share is added to the basic dissipation ratio and limited to between zero and one to obtain the distance reduction parameter. When the distance reduction parameter is equal to zero, the original semantic negative weight is retained. When the distance reduction parameter is between zero and one, the original semantic negative weight is reduced according to the remaining ratio. When the distance reduction parameter reaches one, the corresponding single sentence's reduced negative weight is recorded as zero.

[0068] Starting from the initial boundary of the rhythm imbalance zone, the semantic deviation sequence is read sequentially towards the current accumulation unit. The semantic negative weight of each word is proportionally reduced according to the corresponding distance reduction parameter. Then, the negative weights of single sentences within the same accumulation unit are added together to obtain the single sentence pressure increment. The current single sentence pressure increment is added to the cumulative result of the previous accumulation unit to obtain the current rhythm-semantic pressure index. When entering a new rhythm imbalance zone, the cumulative result is reset. When there are no effective words in the current accumulation unit, its single sentence pressure increment is zero. When the index calculation result exceeds the upper limit of the system's allowed value, it is truncated to the upper limit and the saturation state is recorded.

[0069] Since the semantic negative weight is determined by both the sensitivity intensity of the term and the magnitude of rhythm imbalance, the same term will not produce the same weighting result under smooth expression and obvious rhythm fluctuation. At the same time, the distance reduction parameter gradually weakens the contribution of lexical units that are far from the starting boundary of the rhythm imbalance area, so that the rhythm-semantic pressure index retains the temporal correlation between local acoustic anomalies and adjacent semantic content.

[0070] Step S106, threshold triggering, level determination and policy matching, includes: The current rhythm-semantic pressure index is compared with the intervention trigger threshold. When the current index is less than the intervention trigger threshold, a non-crossing status is generated and the index continues to be updated. When the current index is greater than or equal to the intervention trigger threshold, a crossing status is generated. To reduce the flashing of prompts caused by repeated fluctuations of the index near the threshold, a release threshold lower than the intervention trigger threshold can be set for the exit prompt status. The triggered status is only lifted when the index continuously meets the release conditions or the current rhythm imbalance zone ends.

[0071] When the pressure level has been exceeded, the rhythm-semantic pressure index is compared with multiple non-overlapping pressure level intervals arranged in ascending order of value. Except for the highest level interval, each interval adopts a boundary rule that includes the lower boundary but not the upper boundary. The highest level interval includes both the lower boundary and the upper limit of the system value. When the index is equal to the common boundary of adjacent intervals, it is assigned to the interval with the larger value. When the index exceeds the upper boundary of the highest interval, it is assigned to the highest level. When the index is lower than the lower boundary of the lowest level but has met the trigger threshold, it is assigned to the lowest level. This yields a unique current pressure level identifier.

[0072] Using the current level of pressure, business scenario, and language as search keys, the system searches for corresponding intervention strategy entries in the dialogue degradation strategy library to obtain degradation strategy tags. Strategy entries can indicate interface guidance categories such as reducing the frequency of continuous follow-up questions, switching to neutral confirmation, providing more answer time, splitting complex questions, or transferring to manual review. When multiple entries are found, they are selected in order of scenario matching degree, strategy priority, and version effective time. When no matching entry is found, the preset general mitigation strategy tag is used.

[0073] The crossed-out status identifier, current pressure level identifier, and downgrade policy label are written into the header field of the crossed-out prompt message. The current call identifier, rhythm imbalance zone time range, configuration version, and message generation time are associated with this header to obtain the script downgrade prompt signal. Since the level range adopts a defined boundary assignment rule and the policy query adopts a defined selection order, the same configuration version and the same index input can obtain a unique policy label, which is convenient for stable terminal display and server tracking.

[0074] Step S107: Generate an intervention message package and output visual dialogue guidance information, including: See Figure 3Based on the downgrade prompt signal, an auxiliary intervention instruction is constructed. The message header includes at least the protocol version, message type, message identifier, call identifier, agent identifier, generation timestamp, sequence number, and validity period. The message payload includes at least the index over-limit status identifier, current pressure level identifier, downgrade policy label, policy text resource key, start and end timestamps of the rhythm imbalance zone, and configuration version. The downgrade policy label in the message header is used for routing or quick filtering, and the downgrade policy label in the payload is used for agent terminal parsing and display. If the two are inconsistent, the terminal refuses to execute the message and returns a consistency error.

[0075] The auxiliary intervention command is serialized and integrity verification information is generated to obtain the intervention message packet. Then, the target network address is determined based on the binding relationship between the current call identifier and the agent terminal network address. Before sending, the message routing service verifies that the agent session is still in a valid state. When the network is available, the message is sent through a persistent connection or request-response connection. In the event of a short network interruption, the message to be sent is cached according to the sequence number and retried within the validity period. When the validity period expires, the call has ended, or the agent binding relationship has changed, the message is discarded and the reason for failure is recorded.

[0076] After receiving the intervention message packet, the agent terminal first verifies the protocol version, integrity verification information, call identifier, agent identifier, message validity period, and sequence number, and then parses the degradation policy label and policy text resource key in the payload. Duplicate messages with the same message identifier are processed only once. Expired messages with a sequence number smaller than the processed sequence number are not overwritten with the current prompt. For policy text resource keys that cannot be recognized, a general mitigation prompt is displayed instead of blank content.

[0077] The agent terminal renders visual script guidance information in the floating prompt area of ​​the current call window based on the degradation policy label. The prompt content includes the policy category, the executable script adjustment direction, and the selectable effective time. The display method can be text, color, icon, or a combination thereof. Different pressure levels can correspond to different display priorities, but the terminal does not automatically replace the agent in speaking, nor does it automatically play or send content to the responder. The agent can decide whether to use the prompt based on the call context, thus retaining human judgment.

[0078] After the agent adjusts their subsequent call delivery based on the visual script guidance information, the call access service continues to collect subsequent voice streams, and the ASR speech recognition engine continues to output new speech segments and timestamps. Steps S101 to S107 are repeated from the new pause interval position. When the subsequent window returns to a smooth fluctuation and the rhythm-semantic pressure index meets the release condition, a cancellation prompt message is generated, and the agent terminal cancels or reduces the display priority of the floating prompt, thus forming a closed loop from voice signal analysis to interface prompts and then to the re-analysis of subsequent voice streams.

[0079] Because the intervention message packet carries call identifier, agent identifier, serial number, validity period and configuration version, the message routing service and agent terminal can perform target binding, out-of-order processing, duplicate removal and expiration of messages. At the same time, the floating prompt only provides call adjustment suggestions and does not make decisions that have a substantial impact on individual rights, thereby improving the traceability and controllability of intervention information while maintaining real-time performance.

[0080] Further embodiments of the real-time processing mechanism: At the start of a call, the call access service generates a unique call identifier within the system and establishes a session context that includes the agent channel, the response channel, audio sampling information, the session start time, the current routing node, and the configuration version. After the ASR speech recognition engine accesses the audio stream, it reads the call identifier and associates each speech segment event with the call identifier, channel identifier, and incrementing event sequence number. Only events with the same call identifier and response channel are added to the same analysis queue, thereby avoiding the cross-arrangement of speech segments generated by parallel calls or simultaneous two-way speaking in the pause interval sequence.

[0081] The time base in the session context is zero point at the start of the call. The audio sampling position, ASR start timestamp, and ASR end timestamp are all converted into monotonically increasing time values ​​relative to this zero point. When the access service and the ASR speech recognition engine are located on different computing nodes, the access service adds the segment start sampling position and sampling rate when sending audio segments. The ASR speech recognition engine returns the corresponding sampling position when outputting the segment. The timestamp is converted based on the sampling position first, and the node clock is used only to record the time of message generation and transmission, thereby reducing the impact of node clock deviation on the time difference between adjacent segments.

[0082] When converting timestamps based on sampling locations, the relative number of samples is obtained by subtracting the first sampling location of the call from the target sampling location. Then, the number of seconds relative to the start of the call is obtained by the ratio of the relative number of samples to the sampling rate. When the system needs millisecond timestamps, the number of seconds is multiplied by the number of milliseconds corresponding to 1 second and the millisecond offset is obtained according to the unified rounding rule. The conversion is stopped and the corresponding call segment is marked as invalid when the sampling rate is missing, not greater than 0, or the sampling rate of the same session changes without being declared.

[0083] When determining the time offset between two nodes, the access service records the time when the synchronization request is sent, the ASR node records the time when it receives the request and returns immediately, and the access service records the time when the synchronization response arrives. The access service obtains the local round-trip midpoint by dividing the sum of the request sending time and the response arrival time by two, and obtains a candidate time offset by subtracting the local round-trip midpoint from the ASR node's receiving time. After multiple synchronizations, the candidate time offsets are sorted in ascending order of round-trip duration and the candidate time offset corresponding to the sample with the smaller round-trip duration is selected as the current offset.

[0084] The ASR absolute time is converted to the access service time by subtracting the offset from the current time offset, and then the call start time is subtracted to obtain the session relative time. When the difference between the new candidate time offset and the current offset exceeds the drift tolerance, it is not replaced immediately. Instead, the offset is updated after several consecutive synchronization samples indicate the same direction of change. The update only applies to unconfirmed call segments. Confirmed call segments still use the offset version recorded by the current offset.

[0085] When the ASR speech recognition engine can only return absolute time, it records the time offset between the access service and the ASR node during the session establishment phase, and converts the absolute time to the session time base when receiving segment events. If it is detected that the converted segment start time is earlier than the confirmed time range and the backoff amount exceeds the configured allowable value, the historical index already used for prompting is not directly modified. Instead, the event is written to the queue to be corrected. After receiving a stable version of the same segment, the window that has not yet expired is recalculated from the earliest affected pause position.

[0086] The session termination condition can be that the access service issues a hang-up event, the agent terminal closes the current call window, the audio stream fails to recover after a preset keep-alive timeout, or the upstream explicitly sends a session termination flag. After the session termination condition is met, new call segments are stopped being received, and call segments that have arrived but have not yet stabilized are waited for the maximum convergence timeout. The processing of the last valid window is completed, and then the audio buffer, call segment index and accumulated state are released. At the same time, the necessary audit digest is retained instead of the original voice is stored indefinitely.

[0087] Since all segments, pauses, changes, words, and intervention messages are associated with the same call identifier and unified time base, the local rhythm window can be stably mapped to the current session in the original audio and agent interface. When upstream clock drift or ASR correction occurs, the system limits the correction boundary by sampling location, stable version, and affected range recalculation rules, thereby preventing data that already belongs to other calls or other channels from being incorrectly included in the current rhythm-semantic stress index.

[0088] Version control and stabilization of ASR incremental events are handled as follows: The ASR speech recognition engine can continuously output temporary speech segments, revised speech segments, and stable speech segments. For each speech segment, it maintains the reception status, version number, text status, start and end timestamps, and last update time. When a speech segment is received for the first time, a speech segment record is established. When an event with the same speech segment identifier but a higher version number is received, the new event replaces the old record. When an event with the same version number but a higher stable state is received, the record with the higher stable state is retained. When a late event with a version number lower than the current record is received, only the reason for the lateness is recorded without reverting to the current record.

[0089] The pause interval sequence can be generated after the segment reaches a stable state at the time boundary. Whether the text is stable or not does not affect the preliminary calculation of the pure time structure. If the start timestamp of the next segment has been stabilized while the end timestamp of the previous segment may still be revised, the pause formed by the two is marked as a provisional pause. Provisional pauses can enter the preview window but cannot be directly triggered for intervention. Only after all the pauses that make up the window are converted to the confirmed state can the corresponding convex shape, semantic weighting and threshold crossing results be converted to the state that can be sent.

[0090] When the time boundary of a stable segment undergoes its last revision within the allowable range, the two pause intervals before and after the segment are located based on the segment identifier. The pause values ​​formed by the old time boundary are deleted, and the affected sentence-by-sentence changes and local rhythm triples are regenerated from the revision position backward. The recalculation only extends to the input of several consecutive windows where the revised pause position is no longer included. This maintains the consistency of the time relationship and avoids recalculating the entire call from the beginning for each local revision.

[0091] When the window on which the intervention message was based becomes invalid due to ASR revision, the current oppression level identifier and downgrade policy label before and after the revision are compared. If they are the same, the original message is retained and the internal audit chain is updated. If the triggering conditions are no longer met after the revision, a de-escalation prompt message with a higher sequence number is generated. If the line is still crossed after the revision but the level changes, an update prompt message is generated. The agent terminal only processes messages with higher sequence numbers, so the revision result will not be overwritten by previous messages.

[0092] By managing the time boundary stable state and text stable state of the call segment separately, the system can first use the reliable timestamps to form a rhythm analysis cache, and then complete semantic matching after the text stabilizes, while using the confirmation window as the actual triggering basis. This state separation makes full use of the ASR incremental output and avoids temporary transcription or temporary time boundaries directly causing irreversible agent prompts.

[0093] The sliding window caching, indexing, and incremental geometry processing are implemented as follows: For each call session, a segment queue, a pause queue, a sentence-by-sentence change queue, and a local window queue are maintained. All four types of queues are stored in chronological order and cross-indexed using segment identifiers and pause identifiers. After adding a new acknowledgment segment, the system adds at most one pause interval, adds at most one sentence-by-sentence change after adding a pause interval, and forms at most a new local rhythm triple after adding a sentence-by-sentence change. Therefore, in normal incremental processing, only a limited number of calculations are performed on the tail of the queue.

[0094] Each local rhythm triple records the identifiers of three sentence-by-sentence variations, the horizontal and vertical positions of the three variation nodes, the earliest and latest timestamps covered, the set of constituent segments, the local rhythm arch height, the local rhythm imbalance area, the local rhythm sharpness, the morphological marker, and the configuration version. When any input in the window changes due to ASR revision, the cross-index can directly find the window that references that input and recalculate the derived values ​​of these windows after marking them as invalid.

[0095] When calculating the area of ​​local rhythm imbalance using a piecewise trapezoidal relationship, the directions of two adjacent nodes relative to the time reference axis are read respectively. When two nodes are on the same side of the time reference axis, the area of ​​the trapezoid is formed by the distance from the two nodes to the reference axis and the lateral span. When two nodes are on opposite sides of the time reference axis, the intersection point is first determined based on the linear change relationship between the two nodes, and then the areas of the triangles on both sides of the intersection point are calculated and added together. When a node falls on the reference axis, the distance from the node to the reference axis is treated as zero.

[0096] The local rhythmic arch height, imbalance area, and sharpness use the same configuration version's units and thresholds. If the system switches the timestamp from milliseconds to seconds, the time-related morphological tolerance, area threshold, span threshold, and amplitude reference upper limit are simultaneously calculated when loading the configuration. The configuration loader first checks whether the unit declaration is complete, and then checks whether each threshold is an allowed non-negative value. If any check fails, the previous valid configuration is retained, and partially converted configurations are not allowed to enter the online window.

[0097] To limit the memory usage of long calls, the queue only retains data that can participate in the current unfinished rhythm imbalance zone, the ASR allowed rework range, and the message audit range. When a segment is earlier than all unfinished windows and earlier than the maximum rework time limit, the original audio of the segment can be removed from the fast cache, and only the index digest that is authorized to be saved is retained. The corresponding change amount and word details are released after the current rhythm imbalance zone ends and the review time limit is exceeded.

[0098] The aforementioned incremental caching structure ensures that each new segment drives only a limited number of updates related to the tail of the queue, while the cross-index ensures that boundary revisions only recalculate the affected window. Therefore, the system does not need to repeatedly scan all historical segments during continuous calls, while still maintaining the hierarchical tracing relationship from intervention messages to stress index, semantic weight, local morphology, pause interval, and original segments.

[0099] The continuous representation of local morphological labels and the determination of region boundaries are specifically performed as follows: Each marker in the local morphological marker sequence corresponds to a sliding window with start and end times. Adjacent windows usually share two sentence-by-sentence changes. The markers are processed according to the end time of the window. When a raised marker that first increases and then decreases appears and the previous effective window is a smooth fluctuation marker, the start time of the current window is recorded as the candidate morphological jump time and the candidate rhythm imbalance state is entered. When subsequent windows continue to be raised markers, the candidate state is confirmed and the rhythm imbalance zone is established.

[0100] If a transition mark appears after a raised mark, the rhythm imbalance zone is not terminated immediately; instead, the transition window is placed in the pending buffer. When a raised mark reappears in a subsequent window and the length of the pending buffer does not exceed the allowed number, the pending window is merged into the current rhythm imbalance zone. When a smooth fluctuation mark appears in a subsequent window or the length of the pending buffer exceeds the allowed number, the end time of the last confirmed raised window is taken as the end boundary of the rhythm imbalance zone.

[0101] When multiple local rhythm triples cover the same segment, the tags are processed according to the time intersection. When the segment has a valid intersection with any confirmed rhythm imbalance area, it is assigned the rhythm imbalance area tag. When the segment only intersects with the continuous smooth fluctuation area, it is assigned the normal expression area tag. When the segment crosses both the end boundary of the rhythm imbalance area and the start boundary of the normal expression area at the same time, the segment can be split into two logical segments based on the word-level timestamp, or the physical segment can be kept unsplit but the time range of the two tags can be recorded in the index.

[0102] If a raised marker appears directly after a call that has a missing timestamp, speaker switching, or a long interruption, since there is no available previous smooth window, the start time of the raised window is taken as the region's starting boundary and the boundary source is recorded as a restart boundary. This boundary can still be used for speech extraction and semantic weighting, but it is stored separately from the morphological transition boundary obtained from the transition from smooth to raised in the audit information.

[0103] The set of tagged segments is represented by segment records and segment references. The segment records store the tag type, start and end times, morphological transition times, included window, configuration version, and quality status. The segment references store the original segment identifier and the time range in which it intersects with the segment. This representation supports merging continuous windows into a single analysis segment without changing the boundaries of the original ASR segments. Subsequent secondary transcription, lexical localization, and message auditing can all read the same set of time references.

[0104] The generation of imbalanced response text segments is aligned with lexical time, and the specific operations are as follows: When extracting the response speech segment, the intersecting audio segments are read from the circular audio buffer according to the start and end times of the rhythm imbalance area, and then reassembled according to the sampling position. If the start and end times of the segment are within the segment, the first and last segments are cut off according to the corresponding sampling point. If there is a missing audio segment in the middle, the gap position is recorded in the text segment quality status. Sentences containing gaps do not participate in continuous word accumulation, but sentences before and after the gap that have been confirmed can be retained separately.

[0105] The secondary transcription channel receives audio segments, language identifiers, call scenario identifiers, and segment time offsets, and returns unbalanced response text, sentence-level boundaries, and optional word-level boundaries. The relative time within the transcription channel is added to the segment start time to obtain the word timestamp under the session time reference. Then, based on the original segment time range, the segment to which each word belongs is determined, so that the rhythm imbalance can be transmitted to the semantic weighting process through the segment identifier.

[0106] When reusing the original ASR stable text, the text and time boundaries are directly obtained from the speech segments referenced by the tagged fragment set, and word segmentation is performed on the text within the speech segment; if the original ASR only provides the time of the whole sentence segment and not the word-level time, the system retains the same speech segment identifier according to the order of the words in the sentence, and does not generate precise word-level time out of thin air. Cross-segment phrase matching is only allowed to be performed when the two speech segments are consecutive in time and the pause in between does not exceed the phrase continuity threshold.

[0107] Text normalization preserves the original characters that are meaningful for the dictionary matching, while establishing a character position mapping from normalized text to original text; conversion of uppercase and lowercase, full-width and half-width, traditional and simplified Chinese or number format only applies to the matching copy, and the matching result still points to the original word and original time range through position mapping. Therefore, when the agent terminal or audit module needs to display context, it can read the original expression, while the semantic deviation sequence uses a unified word form.

[0108] When establishing the character position mapping from normalized text to original text, the characters are processed in the order of the original characters. The start and end positions of each original character are recorded. When an original character is converted into a normalized character, the normalized character is set to point to the current range of the original characters. When an original character is converted into multiple normalized characters, all the normalized characters are set to point to the current range of the original characters. When multiple original characters are merged into a normalized character, the normalized character is set to point to the combined range from the start position of the first original character to the end position of the last original character.

[0109] When performing word segmentation on standardized Chinese text, starting from the current unprocessed character position, all candidate words starting from that position are searched in the standard word forms of the preset basic dictionary and the high-pressure sensitive word database. The candidate word with the most covered characters is determined as the current word element. Candidate words with the same number of covered characters are selected according to the fixed order formed by the entry identifier of the basic dictionary and the entry identifier of the high-pressure sensitive word database. If no candidate word is found, the single Chinese character, continuous letter string, or continuous number string at the current position is determined as the current word element.

[0110] After determining the current word element, the processing position is moved to the character after the end position of the current word element and the search continues until the end of the normalized text is reached. Punctuation marks, newlines, and ASR sentence-level boundaries are used as word element boundaries and no sensitive word elements are generated. Spaces only separate adjacent letter strings or number strings. Each word element obtained by segmentation saves the normalized start and end positions, and then obtains the corresponding original text range through character position mapping.

[0111] When the same normalized character is obtained by merging multiple original characters or when one original character is expanded into multiple normalized characters, the original range of the word may cover additional characters. The original range of the word is the minimum start and maximum end of the entire original range referenced by the normalized start and end positions. If the position mapping is missing or the start is greater than the end, the system marks the word as invalid and prohibits it from participating in cross-segment phrase matching.

[0112] When ASR confidence information is available, lexical units below the text availability threshold can be marked as pending review without being assigned sensitive word weights, while lexical units above the availability threshold participate in matching normally. This confidence threshold only determines whether the text enters the semantic calculation and does not change the local rhythmic pattern. Therefore, even if there is no available text in a certain imbalance area, the system still retains its acoustic label but does not automatically add negative semantic weights due to unknown text.

[0113] Because the unbalanced response text segment is aligned with the local rhythm region through segment start offset, original segment identifier and word position mapping, the word matching result can locate the actual audio time range; missing audio, low confidence text and no word-level timestamp are all processed with explicit degradation rules, thereby avoiding treating unconfirmed word content as a definite semantic participation pressure index.

[0114] The construction, version management, and deterministic matching of the high-pressure sensitive word database are carried out as follows: The high-pressure sensitive word library can be constructed from authorized business corpora and manual review results. The library management terminal first performs deduplication and normalization on candidate word elements or phrases, and then configures each entry with word identifier, standard word form, identifiable variant, applicable language, applicable scenario, preset sensitivity intensity, activation status and version number; the preset sensitivity intensity is used to indicate the relative attention of the entry in the call assistance scenario, and does not indicate the medical judgment of the speaker's psychological state.

[0115] When the thesaurus is published, an immutable version snapshot is generated. The term identifiers and sensitivity intensities within the same version remain unchanged. Adding, deleting, or modifying entries will all form a new version. The current thesaurus version is recorded at the beginning of the rhythm imbalance zone and is used continuously until the end of that zone. Even if the management releases a new version in the middle of processing, it will start loading from the next rhythm imbalance zone, thus ensuring that the matching and weighting benchmarks within the same cumulative range are consistent.

[0116] The matcher can be implemented using a prefix tree, a finite state automaton, or a phrase table sorted by length. During processing, it scans normalized words in the order of text time, first collecting candidate phrases that can be completely covered starting from the current position, and then sorting them from most to least covered words. If the coverage lengths are the same, the preset sensitivity intensity is compared. If the sensitivity intensity is still the same, the fixed order of the word identifiers is compared. After selecting the first candidate entry, the scanning position is moved to the end position of that entry.

[0117] For multiple non-overlapping matching entries, semantic deviation items are generated separately. Each item saves the entry identifier, covered word element, original text position, start and end time, corresponding discourse segment, preset sensitivity intensity, rhythm imbalance magnitude, and semantic negative weight. For word element entries whose original text position has been occupied by long phrases, deviation items are not generated again, thereby ensuring that the same text and the same dictionary version get a unique matching sequence.

[0118] When a dictionary entry is deactivated but the old version is still used in an unfinished rhythm imbalance zone, the old version is allowed to complete the current segment and the version number is retained in the audit log. The next newly created segment will no longer use the deactivated entry. When the dictionary version file is corrupted, the signature verification fails, or the sensitivity strength is lacking, the system does not replace it with a zero value, but continues to use the previous valid version or suspends semantic authorization and generates a configuration anomaly alarm.

[0119] The thesaurus management terminal can periodically extract anonymized or de-identified false trigger samples for authorized personnel to review. Based on the review results, the entries, scenario scope, or sensitivity intensity can be adjusted and a new version can be released. However, this embodiment does not use unproven accuracy, recall, or improvement ratios to prove its effectiveness. Versioning, deterministic overlap processing, and failure rollback are only used to ensure that the word matching process is repeatable, traceable, and can be implemented by the deployment personnel.

[0120] The specific steps for mapping the amplitude of rhythm imbalance and generating semantically negative weights are as follows: The original rhythm imbalance magnitude is obtained from the absolute deviation of the intermediate change amount relative to the first and last change amounts, respectively. The average of the two absolute deviation amounts can be taken so that the deviations on both sides are included in the magnitude representation. In implementations that require maintaining constraints on the weaker side, the smaller of the two absolute deviation amounts can also be taken. However, only one magnitude rule can be selected in the same configuration version, and the magnitude rule identifier is written into each semantic deviation item.

[0121] The amplitude mapping configuration includes at least the input time unit, reference lower limit, reference upper limit, output range, and out-of-bounds handling method. When the original amplitude is less than or equal to the reference lower limit, the lower boundary of the output mapping range is output. When it is between the reference lower limit and the reference upper limit, it maintains a monotonic order according to its relative position. When it reaches or exceeds the reference upper limit, the upper boundary of the output mapping range is output. When the reference upper limit and the reference lower limit are equal, a valid mapping cannot be established, and the configuration loader rejects the configuration.

[0122] When the original rhythm imbalance amplitude is between the lower and upper reference limits, first subtract the lower reference limit from the original rhythm imbalance amplitude to obtain the amplitude position difference, then subtract the lower reference limit from the upper reference limit to obtain the reference span, and use the ratio of the amplitude position difference to the reference span to obtain the relative amplitude position; then subtract the lower boundary of the output interval from the upper boundary of the output interval to obtain the output span, multiply the relative amplitude position by the output span and add the lower boundary of the output interval to obtain the rhythm imbalance amplitude used for weighting.

[0123] Before performing amplitude mapping, check that the original rhythm imbalance amplitude, reference lower limit, reference upper limit, and output interval boundaries use compatible data types. Check that the reference upper limit is greater than the reference lower limit and that the output interval upper boundary is not less than the output interval lower boundary. If any check fails, the output amplitude mapping is invalid. When the original amplitude is exactly equal to the reference lower limit, the output interval lower boundary is reached. When it is exactly equal to the reference upper limit, the output interval upper boundary is reached. When it exceeds the reference range, it is truncated according to the corresponding boundary.

[0124] When amplitude mapping is implemented using a segmented mapping table, each mapping entry includes at least the lower boundary of the input interval, the upper boundary of the input interval, the output reference value, the version number, and the valid status. First, the entries are arranged in ascending order according to the lower boundary of the input interval. The first valid entry containing the original rhythm imbalance amplitude is searched, and the common boundary is assigned to the interval with the larger input value. If no entry is found, the lowest output value, the configured gap back value, or the highest output value is used according to whether the value is lower than the minimum interval, falls into the interval gap, or is higher than the maximum interval.

[0125] If the segmented mapping table configures the output reference values ​​at both ends of each input interval, first determine the relative position of the original rhythm imbalance amplitude within the input interval, and then perform linear interpolation on the output reference values ​​at both ends based on this relative position. The specific process is consistent with the upper and lower limit mapping process. When the mapping table entries overlap, the new version is not loaded. When the entries are repeated, the entry with the later version order is retained, thereby ensuring that the same original amplitude only gets one weighted amplitude.

[0126] When a segment to which a word belongs is covered by multiple raised windows, candidate windows are first filtered based on the intersection of the word's time range and the time ranges of each window. Then, the window with the longest intersection duration is selected. If the intersection durations are the same, the window with the larger original rhythm imbalance amplitude is selected. If they are still the same, the window with the earlier end time is selected. This order ensures that the amplitude source under the overlapping window is unique and that the specific local rhythm triplet can be found by looking up the word.

[0127] The semantic negative weight is determined by the pre-set sensitivity intensity of the term and the rhythm imbalance amplitude used for weighting. When both are non-negative, the weight is also non-negative. If the term sensitivity intensity is missing, the amplitude mapping state is invalid, or the term does not have a definite segment to which it belongs, no valid semantic negative weight will be generated and the reason for failure will be recorded. The system will not use random values, global average values, or the amplitude of the previous term to fill these missing inputs.

[0128] For multiple non-overlapping matching entries in the same sentence, retain the weight of each word and add them together when forming the single sentence pressure increment; in order to avoid the value from increasing indefinitely due to abnormally long text or repeated transcription, an upper limit associated with the configuration version can be set for the single sentence pressure increment. Once the upper limit is reached, the sentence is marked as saturated, and the upper limit value is used for subsequent accumulation, but all matching details are still retained for auditing.

[0129] Since the sensitivity intensity of a word provides the relative degree of semantics, and the amplitude of rhythmic imbalance provides the acoustic degree of the time region in which the word is located, the combination of the two causes the same word to have different weights in different rhythmic regions; the amplitude rules, overlapping window selection and missing input processing all have a definite order, so the semantic deviation sequence can be stably reproduced by the same set of inputs.

[0130] Distance reduction, sentence-by-sentence accumulation, and exponent status management are implemented as follows: The cumulative range is established based on the rhythm imbalance zone. The range is divided into continuous cumulative units according to the sentence-level boundary or stable segment boundary, and each unit is assigned a position index that increases from zero. The cumulative unit to which the same word belongs is determined by the intersection of its time range and the unit's time range. If it crosses two units and the intersection is the same, it is assigned to the unit with the later time, so as to ensure that each semantic deviation item only participates in the single sentence pressure increment once.

[0131] The time distance step represents the number of units crossed between the current accumulation unit and the starting unit of the rhythm imbalance zone, rather than the node message transmission time or absolute number of seconds; the basic dissipation ratio represents the fixed share that needs to be deducted even if it is located in the starting unit; the step increment decay coefficient represents the dissipation share that increases for each accumulation unit away from the current unit; the sum of the two is limited to between zero and one to form the distance reduction parameter.

[0132] The distance reduction parameter is applied to each semantic negative weight in the current unit to obtain the sentence reduced negative weight. The reduction results in the same unit are then added together to form the sentence pressure increment. The current rhythm-semantic pressure index is obtained by sequentially accumulating all sentence pressure increments from the segment start unit to the current unit. The system saves the current cumulative value and the contribution details of each unit. Therefore, the subsequent cumulative value can be recalculated from the affected units when ASR is revised or the dictionary configuration is rolled back.

[0133] If the current accumulated unit has no matching word, only low-confidence word, or a gap in speech, the single-sentence pressure increment is zero and the position index is retained. The time distance step of subsequent units still increases according to the actual number of units. This processing will not treat missing semantics as negative semantics, nor will it delete missing units, causing the distance between subsequent words and the segment start boundary to be artificially shortened.

[0134] After the rhythm imbalance zone ends, the final index and contribution details of that zone are frozen, and the words in the subsequent normal expression zone are no longer added to the ended zone. When a new rhythm imbalance zone is established, the calculation starts from zero again, but the previous prompt state can be retained on the interface until the release condition is met. The reset of the calculation zone and the removal of the interface prompt are controlled by the zone boundary rules and the prompt state machine, respectively.

[0135] When the accumulated value approaches the upper limit of the system's numerical type, the remaining available range is checked before the next accumulation. If the accumulation is expected to exceed the limit, the exponent is truncated to the configured upper limit and a saturation state is set. The level determination module assigns the saturation exponent to the highest valid level. The audit record also retains the contribution of each unit before truncation to avoid incorrectly lowering the prompt level after the value overflows and becomes negative or small.

[0136] The distance reduction parameter incorporates the position of the lexical unit relative to the starting boundary of the rhythm imbalance zone into the weight contribution, and sentence-by-sentence accumulation preserves the superposition relationship of continuous semantic items to the current state; through unit indexing, zero increment retention, segment reset and overflow truncation, it can clearly handle boundary cases such as empty sentences, missing text, new segments and extreme cumulative values.

[0137] The intervention trigger and prompt state machine operates as follows: Each call session corresponds to a prompt state machine. The initial state is untriggered. When the current rhythm-semantic pressure index is less than the intervention trigger threshold, it remains in the untriggered state. When the index reaches or exceeds the intervention trigger threshold, it changes to the triggered state and generates a speech downgrade prompt signal. After entering the triggered state, it continues to update the current level according to the pressure level range of the subsequent index, instead of repeatedly creating the same level of prompt message for each new word.

[0138] In the triggered state, when the current level is higher than the most recently sent level, an upgrade message is generated and carries a new downgrade policy label. When the current level is the same as the most recently sent level, only the server-side cumulative status is updated. When the current level is reduced but the release conditions have not yet been met, a downgrade update message can be generated or the current prompt can be kept until the minimum display duration expires. The specific method is determined by the prompt status configuration and is fixed in the same session.

[0139] Release conditions may include at least one or a combination of the following: the current rhythm imbalance zone ends, several consecutive windows show smooth fluctuations, and the current index is below the release threshold. The judgment is made according to the configured logical relationship. When the release threshold is lower than the intervention trigger threshold, a hysteresis interval is formed. The index does not repeatedly switch between trigger and release states within the hysteresis interval, thereby reducing the frequent flashing of prompts on the agent terminal.

[0140] When the compression level intervals are loaded, they are sorted from smallest to largest according to their lower boundaries and the system checks for overlaps or gaps. If there are overlaps, the configuration is invalid. If there are gaps, the configuration declares that the gap belongs to the previous level, the next level, or the general level. In the preferred implementation of this embodiment, each interval is continuous from beginning to end and adopts the rule that the lower boundary includes the interval and the upper boundary does not. The highest interval includes the upper limit of the system index.

[0141] When determining the intervention trigger threshold, the calibration program collects authorized historical accumulated units that have been manually labeled to indicate whether seat adjustment is required. It calculates the rhythm-semantic pressure index of each accumulated unit according to online rules, sorts all non-repeating indices from smallest to largest, and uses the median value of adjacent indices as a candidate trigger threshold. For each candidate threshold, the calibration program determines the accumulated units with indices greater than or equal to the candidate threshold as triggers, and separately calculates the false trigger ratio of normal units and the recognition ratio of units to be adjusted. Finally, it selects the final intervention trigger threshold in the order of the largest false trigger constraint, recognition ratio, and threshold.

[0142] When constructing multiple oppression level intervals, the calibration program selects samples with indices greater than or equal to the intervention trigger threshold, sorts these indices from smallest to largest, and divides the sorted samples into several groups with consecutive order and a difference of no more than one in the number of samples according to the preset number of levels. Between two adjacent groups, the sum of the largest index of the previous group and the smallest index of the next group is divided by 2 to obtain the common boundary value. The lower boundary of the lowest level is taken as the intervention trigger threshold, and the upper boundary of the highest level is taken as the upper limit of the system index, thus forming oppression level intervals that are consecutive from beginning to end.

[0143] When the number of samples reaching the intervention trigger threshold is less than the preset number of levels, the calibration program reduces the number of available levels and outputs a level shortage status, or maintains the previous effective level interval version without constructing an empty interval; when the maximum index of the first group and the minimum index of the second group of two adjacent groups are equal, the equal index is directly used as the common boundary value, and the level is assigned to the higher level according to the rule of the common boundary, without repeatedly establishing two levels with the same upper and lower boundaries.

[0144] The strategy matching key can include the current pressure level identifier, business scenario identifier, language identifier, and agent skill group identifier. The strategy library first filters candidate items with the same level, and then selects them in sequence according to scenario exact matching, language matching, skill group matching, and strategy priority. If there are no completely matching items, the non-core conditions are relaxed layer by layer. If there are still no items, the general mitigation strategy label is returned. The level identifier itself must not be rewritten due to the lack of strategy items.

[0145] The script degradation strategy library is implemented using a strategy entry table. Each entry includes a strategy entry identifier, pressure level identifier, business scenario identifier, language identifier, agent skill group identifier, strategy priority, degradation strategy tag, strategy text resource key, version number, effective time, expiration time, and enabled status. When loading entries, the existence of the degradation strategy tag and strategy text resource key is first checked, and then the effective time is checked to be earlier than the expiration time. For duplicate entries with the same matching key and priority in the same version, only the entry with the later version generation order is retained.

[0146] When querying policy entries, first filter entries that are enabled and within the validity period using the current oppression level identifier. Then compare the business scenario, language, and agent skill group with the current session fields respectively. For each field that is equal, increase the match count of the entry by one unit. The services are sorted in the following order: match count from largest to smallest, policy priority from largest to smallest, effective time from latest to earliest, and policy entry identifier from smallest to largest. The downgrade policy label of the first entry in the sorted list is selected as the query output.

[0147] The system records the state before the state machine transitions, the state after the state transition, the trigger index, the level range, the policy label, the configuration version, and the message sequence number for each state transition. Therefore, when an ASR revision, policy update, or network retry occurs, the system can distinguish between a new state change and the repeated transmission of the same state message, and enable the agent terminal to eventually converge to the latest valid state on the server.

[0148] Intervene message protocols, reliable transmission, and terminal display; specific operations are as follows: Intervention message packets adopt a header and payload partitioning structure. The header is used for protocol parsing, target routing, and integrity verification, while the payload is used to express the current analysis status and display content. The message type is at least distinguished as trigger, level update, policy update, and cancellation. The protocol version determines the field interpretation method. When the terminal does not support the current protocol version, it returns an unsupported status and retains the existing valid prompts, and does not directly display unparseable data to the agent.

[0149] When serializing auxiliary intervention instructions, the message type, message identifier, call identifier, agent identifier, generation timestamp, sequence number, validity period, index overrun status identifier, current oppression level identifier, downgrade strategy label, strategy text resource key, rhythm imbalance zone start and end timestamps, and configuration version are read sequentially according to the field order specified in the protocol version. Integer and time fields are converted to fixed-byte length high-byte-first encoding, boolean status is converted to preset single-byte status code, and text fields are converted to byte sequences with unified character encoding.

[0150] For text fields with variable length, first determine the number of bytes in the text byte sequence, encode the number of bytes into a fixed-length field length prefix, and then place the field length prefix before the text byte sequence. After each field is encoded, concatenate them in the order of the protocol fields to obtain the payload byte sequence. Then, encode and concatenate the protocol version, message type, message header length, payload length, and routing fields into a message header byte sequence, and place the message header byte sequence before the payload byte sequence to obtain the byte sequence to be verified.

[0151] During deserialization, the agent terminal first reads the fixed-length protocol version and message header length, determines the starting point of the payload based on the message header length, and then reads each fixed-length field in sequence. When reading text fields, it first parses the field length prefix, then reads the corresponding number of bytes from the current position and restores the text according to the unified character encoding. If the field length exceeds the remaining message length, the field order is inconsistent with the protocol version, or there are undeclared extra bytes after parsing, the message is marked as invalid and display processing is stopped.

[0152] The message identifier is associated with the call identifier and state transition event when it is generated. The network retry for the same state transition reuses the same message identifier without creating a new identifier. The sequence number monotonically increases within the same call session. The message routing service can repeatedly deliver unacknowledged messages. The agent terminal deduplicates messages by using the message identifier and determines the newness by using the sequence number. Only messages with a sequence number higher than the last processed sequence number can change the current prompt status.

[0153] The validity period of a message is determined based on whether the current call is still ongoing and the time sensitivity of the prompt content. The routing service verifies the validity period before sending the message and before the terminal displays it. If the call has ended, the agent has switched sessions, or the validity period has expired when the message arrives, the terminal discards the message and returns an expired status. After receiving the expired status, the server will not retry, thus avoiding the previous call's policy prompt from entering the next call interface.

[0154] Integrity verification can use message digests or authentication codes with keys. Routing connections use access tokens or two-way authentication to determine the identities of the server and agent terminals. If integrity verification fails, the target agent does not match, or the call identifier does not match, the terminal refuses to parse the payload and records the security event. The message text resource key is only read from the local controlled resource table or the server's controlled resource interface after verification.

[0155] In one implementation of keyed authentication codes, the key management service allocates a key identifier and an authentication key for the current protocol version, performs a standard keyed hash operation on the byte sequence to be verified to obtain a fixed-length authentication code, and appends the key identifier and authentication code to the end of the message; the agent terminal obtains the corresponding authentication key based on the key identifier, re-executes the same keyed hash operation on the received message header and payload, and compares the newly obtained authentication code with the authentication code carried in the message byte by byte. If all bytes are equal, the integrity check is passed.

[0156] When comparing authentication codes byte by byte, the agent terminal initializes the difference accumulation value to zero, performs a bitwise XOR operation on the bytes at the same position of the two authentication codes, then performs a bitwise XOR operation on the difference accumulation value and updates the difference accumulation value. After traversing all authentication code bytes, if the difference accumulation value is still zero, they are considered equal; otherwise, they are considered unequal. If the authentication codes have different lengths, the key identifier does not exist, or the key has expired, the verification is directly deemed to have failed, and the policy text resource key is not parsed after the verification fails.

[0157] The message routing service maintains a call binding table, with entries including call identifier, agent identifier, terminal connection identifier, network address, binding version, effective time, and valid status. During routing, the service first searches for valid entries using the call identifier, then compares the agent identifier in the entry with the agent identifier in the message header. If they match, the terminal connection identifier and network address are obtained as the target. If multiple valid entries are found, the entry with the higher binding version is selected. If the versions are the same, the entry with the later effective time is selected.

[0158] When no valid call binding entry is found, the message routing service places the message in the waiting queue and records the first waiting time. Upon receiving a new binding event, it re-queries according to the call identifier. If the waiting time exceeds the message validity period, the message is deleted and the routing failure status is returned. If the found network address is unreachable, it first tries the backup address corresponding to the same terminal connection identifier. If the backup address is still unreachable, it enters the retry queue.

[0159] When calculating the retry waiting time, the message routing service uses the basic retry interval as the first waiting time. For each additional retry, the previous waiting time is multiplied by a preset growth factor, and the resulting time is compared with the maximum retry interval. If the maximum retry interval is exceeded, the maximum retry interval is used. Before each retry, the routing service re-verifies the message validity period, call status, and binding version. Retrying stops when the maximum number of retries is reached or the message has expired.

[0160] The floating notification area is bound to the current call window and displays information including policy category, operation direction, optional example prompts, and validity status. The interface can change the border, icon, or display priority according to the level, but color is not the only information carrier. The terminal provides text descriptions, keyboard accessibility, and screen reading support for the prompts. Agents can confirm, temporarily hide, or report that it is not applicable. Confirmation does not change the algorithm index but serves as a record for subsequent manual review.

[0161] When the agent terminal renders the visual script guidance information, it first queries the local resource table using the strategy text resource key. The resource table entries include resource key, language identifier, title text, body text, icon identifier, style identifier, and version number. When multiple entries are found, the entry with the same language as the current session is selected first, and then the entry with the higher version number is selected. If the corresponding language is not found, the general language entry is used. If the resource key is not found, the general softening prompt entry is used.

[0162] The terminal reads the background color, foreground color, border style, icon size, padding, and display priority from the style table based on the style identifier. It then creates an independent prompt layer on the current call window and places the title text, body text, and icon into the prompt layer in a fixed layout order from top to bottom. When the width of the prompt layer exceeds the available width of the call window, it wraps to the available width without shrinking to below the minimum readable font size. When the height exceeds the available height, it displays the body scrolling area while keeping the title and close control visible.

[0163] When a higher-sequence-numbered level update or policy update message is received during the same call, the terminal reuses the existing prompt layer and replaces the title, body, icon, and style. When a cancellation message is received, the terminal first checks that the cancellation message sequence number is higher than the current prompt sequence number, and then cancels the prompt layer or changes its display status to cancelled. When the message validity period expires and no update is received, the terminal cancels the prompt layer according to the validity period timer and returns the final display status to the message routing service.

[0164] When a terminal reconnects after going offline, it requests an incremental message from the routing service with the sequence number of the last processed call for each active call. The server only returns the latest status with a higher sequence number that is still valid. If the missing intermediate message has been overwritten by a higher sequence number status, it can send only a snapshot of the current status. The terminal applies the snapshot and updates the last processed sequence number, thus avoiding the repeated playback of the invalid message after the connection is restored.

[0165] Concurrent scheduling, resource control, and real-time performance assurance are implemented as follows: Events are partitioned according to call identifiers. Events for the same call are always updated serially by the same logical processing queue. Different calls can be processed in parallel by different worker threads or different service instances. When instance migration occurs during partition routing, the old instance first submits the most recently processed sequence number and state snapshot of the call. The new instance reads the snapshot from the state storage and continues to process events with higher sequence numbers, avoiding two instances modifying the same rhythm - semantic pressure index at the same time.

[0166] In one implementation of a partitioned index, the partitioned router first converts the call identifier into a byte sequence according to a unified character encoding, initializes the hash accumulation value to a preset non-negative initial value, and then processes each byte in byte order. For each byte processed, the current hash accumulation value is multiplied by a preset hash multiplier, and the unsigned value of the current byte is added. The resulting sum is limited to a preset integer bit width. After processing all bytes, the call identifier hash value is obtained.

[0167] The partition router uses the remainder obtained by dividing the call identifier hash value by the current number of valid partitions as the partition index. The remainder starts from zero and is less than the number of valid partitions. When the number of valid partitions is zero, the remainder operation is not performed and the state of no available partitions is returned. When the call identifier is empty or the character encoding fails, the event is written to the invalid event queue. The partition version is fixed during the processing of the same call session. Expansion or contraction takes effect from the next session or after the state migration is completed.

[0168] During state migration, the old instance writes the session snapshot along with the partition version, the last processed event sequence number, and the snapshot content summary to the state store. The new instance reads the snapshot, recalculates the content summary, and compares them. Only when the summaries match and the partition version meets the migration target will the new instance take over the event. After the new instance successfully takes over, it writes a takeover confirmation. When the old instance receives the confirmation, it stops processing events with higher sequence numbers. If it does not receive the confirmation, it retains the processing rights of the old instance and avoids double writing.

[0169] The processing tasks for a single session are categorized into timestamp events, text stabilization events, configuration events, and terminal confirmation events. Timestamp events are prioritized for pause and window calculations. Text stabilization events are processed into lexical units when the corresponding rhythm imbalance zone exists. Configuration events only update the next applicable boundary. Terminal confirmation events are asynchronously written to the audit log. This priority relationship ensures that the critical data chain required to form real-time prompts is not blocked for a long time by low-priority recording tasks.

[0170] When the system load increases, the service prioritizes the calculation of the time structure of confirmed segments, the text processing of segments with rhythm imbalance, and the delivery of messages that have crossed the line, while delaying the detailed semantic analysis of normal expression segments, unnecessary statistics, and offline audit summaries. If the load continues to exceed the capacity, the system returns a resource shortage status to new sessions and the upstream performs degradation, instead of randomly discarding certain sensitive words or certain cumulative units in already started sessions.

[0171] Each processing stage records the event reception time, stage start time, stage end time, and message sending time to calculate the processing latency between the stable segment and the terminal display. This embodiment does not fabricate specific latency values. Deployers can set stage timeouts and total latency budgets according to business requirements. When a stage exceeds the budget, the timeout status is recorded and the unavailable branch is skipped or the non-triggered status is output. Real-time prompts are not forcibly generated based on expired analysis results.

[0172] The audio cache capacity is configured based on the maximum allowed repair time limit, the maximum duration of the rhythm imbalance zone, and the number of concurrent sessions. Segment and lexical indexes use bounded queues, and audit logs are written asynchronously in batches. When the memory limit of a single session is reached, the system first releases the details of normal segments that have been confirmed not to participate in the repair, then releases the expired audio cache. If it still cannot be recovered, the real-time analysis of the session ends and a resource anomaly record is generated.

[0173] Serial updates within the same call ensure the sequential relationship between pauses, windows, terms, and exponent states, while parallel updates across different call partitions ensure service scalability. Bounded caching, task priorities, and load degradation rules further clarify which critical processes are retained when resources are insufficient, thus enabling the real-time auxiliary chain to maintain deterministic data dependencies in a concurrent environment rather than relying on unordered parallel results.

[0174] The specific steps for anomaly recovery, audit trail, and configuration rollback are as follows: The session state snapshot is periodically saved. The snapshot includes the sequence number of the last confirmed call segment, the call segment record still within the repair range, the tail of the pause and change queue, the unfinished rhythm imbalance zone, the current accumulation unit, the current index, the prompt state machine state, and the sequence number of the last sent message. After the service restarts abnormally, the most recent snapshot is read, and the events after the snapshot sequence number are replayed from the event storage. The replayed messages use the original event identifier to maintain idempotency.

[0175] If a snapshot is corrupted or inconsistent with the call identifier in the event store, the system does not continue to accumulate using the incomplete state, but instead reconstructs the time structure of the call from the earliest available stable segment. When it is impossible to obtain enough segments to form a triple, it returns to the waiting state. Historical overrun results that have exceeded the message validity period are only written to the audit log and are not resent to the agent terminal.

[0176] The configuration center saves a content summary and the previous valid version for each version. When loading a new version, it checks the field integrity, unit consistency, threshold order, interval continuity, ratio range, thesaurus signature, and policy mapping availability in sequence. Only after all checks pass can the new version be marked as available. If any check fails, the previous valid version will continue to be used, and the failed fields, version number, and check time will be written to the configuration audit log.

[0177] The audit chain uses the intervention message as the output endpoint and records the prompt state transition, stress index, single-sentence stress increment, semantic deviation, lexicon entries, rhythm imbalance amplitude, local rhythm triplet, sentence-by-sentence change, pause interval, and original segment identifier referenced by the message. Audit queries return the necessary levels according to permissions. Ordinary agents can only view the current prompt, authorized reviewers can view the de-identified calculation chain, and system administrators can view runtime and configuration errors.

[0178] When a manual review confirms that a certain prompt is not applicable, the review result is recorded as an independent feedback record associated with the message identifier and configuration version. It does not directly rewrite the original speech segment, word matching, or index value that has already occurred. When it is necessary to adjust the thesaurus, threshold, or strategy, a new version is generated through the configuration management process and used from the specified effective boundary, so that historical results can be reproduced according to the configuration at that time.

[0179] Anomaly recovery, configuration rollback, and audit trail work together to ensure that existing calculations are not silently altered when online services are restarted, misconfigured, or manually reviewed. The system explicitly moves unrecoverable data to a waiting, invalid, or non-triggered state and preserves processing boundaries through version numbers and event sequence numbers, enabling subsequent technical personnel to pinpoint the specific failure point.

[0180] Before going live, test data can be constructed that includes smooth pauses, convex pauses with initial increases followed by decreases, missing timestamps, overlapping segments, ASR boundary revisions, empty text, overlapping phrases, equal thresholds, common interval boundaries, network duplicate delivery, and terminal disconnection recovery. The test data is used to verify the established input-output relationships and boundary processing, and is not used as experimental evidence of real user psychological states or system performance improvements.

[0181] The time structure test sequentially verifies the paragraph ordering, pause intervals, sentence-by-sentence variation, triplet nodes, arch height, imbalance area, sharpness, morphological markers, and rhythm imbalance zone boundaries. The test tool retains both expected and actual values ​​for each intermediate result. When encountering a window with an invalid timestamp, the expected result should be to skip or wait for revision, rather than generating an undefined value.

[0182] The semantic processing test verifies the text normalization location mapping, longest phrase priority, sensitivity intensity priority for the same length, word identifier order decision, word segment to which the word belongs, amplitude mapping, semantic negative weight, distance steps, reduction parameters, single sentence pressure increment and cumulative index; when the same input, the same dictionary version and the same configuration version are executed repeatedly, the intermediate results should be consistent.

[0183] The strategy and message testing sets the index below the trigger threshold, equal to the trigger threshold, within each level range, outside the common boundary and the highest upper limit, respectively, to check the overstepping status, boundary attribution, strategy label and message type; then, it performs duplicate delivery, out-of-order delivery, expired delivery and target seat mismatch tests on the message, and the terminal should respectively show deduplication, rejection of old sequences, discarding of expired and rejection of wrong targets.

[0184] The closed-loop test starts with the audio segment input, passes through ASR events, rhythm windows, semantic matching, exponential accumulation, and policy messages to reach the agent terminal, and then sends the subsequent segments into the next cycle to check whether the prompt status is released according to the subsequent smooth window and release conditions. If any stage in the closed loop lacks necessary input, the system should output the corresponding quality status and stop subsequent actions that depend on that input, instead of using simulated success results to cover up the gap.

[0185] The online check also includes data authorization scope, audio caching period, log anonymization, role permissions, configuration approval, prompt text, manual review path and abnormal shutdown switch. After the check is passed, it will be enabled gradually in small-scale sessions. In the absence of real verification data, this embodiment only describes the executable check process and does not claim specific accuracy, recall, latency reduction ratio or business conversion effect.

[0186] Table 1 lists the main configuration items, dimensions or value forms, and acquisition methods in this embodiment. The specific production values ​​are determined by the authorized samples and false trigger constraints in the deployment scenario, and are not limited to this application by unfounded fixed values.

[0187] Table 1 is a structure table showing the main configuration items, their dimensions or value forms, and how they are obtained. ; In addition to the rules listed in Table 1, the following boundary processing is also performed: when the number of segments is insufficient to form three consecutive sentence-by-sentence changes, only the data is cached and no morphological conclusion is output; windows with missing timestamps are not included in the judgment; negative pause intervals are truncated according to the overlap tolerance or wait for ASR revision; sharpness is not calculated when the span is zero; the semantic negative weight is zero when there is no match in the lexicon; the rhythm-semantic pressure index is zero when the cumulative range is empty; a general mitigation strategy label is used when the strategy mapping is missing; and the delivery is stopped and the reason is recorded when the terminal address is invalid or the message expires.

[0188] like Figure 2 As shown, embodiments of the present invention also provide a real-time call assistance analysis system based on ASR, comprising: The acquisition module is used to acquire the speech segment sequence segmented by the speech recognition engine according to the timestamp, calculate the timestamp difference at the boundary of adjacent speech segments, and obtain the pause interval sequence. The extraction module is used to calculate the sentence-by-sentence change of the pause interval between the current segment and the previous segment based on the pause interval sequence, and obtain the rhythm fluctuation sequence; extract the fluctuation pattern of the three sentence-by-sentence changes arranged continuously along the time axis in the rhythm fluctuation sequence to construct the local pause fluctuation trend. The "Classification" module is used to classify segments with local pauses and fluctuations that show an initial increase followed by a decrease in a convex shape into the rhythm imbalance zone, and segments with smooth fluctuations into the normal expression zone, thus obtaining a set of labeled segments. The transcription module is used to extract the corresponding response speech segments from the rhythm imbalance area in the set of labeled segments and transcribe them to obtain the imbalanced response text segments; extract the word units in the imbalanced response text segments and match them with the preset high-pressure sensitive word library, assign semantic negative weights according to the product of preset sensitivity intensity and rhythm imbalance amplitude, and construct semantic deviation sequences. The calculation module is used to define the cumulative range segment by segment within the rhythm imbalance zone, and accumulate the semantic negative weights in the semantic deviation sequence sentence by sentence to obtain the rhythm-semantic pressure index; determine whether the rhythm-semantic pressure index exceeds the preset intervention trigger threshold, trigger the generation of the speech downgrade prompt signal; and match the corresponding downgrade strategy label according to the pressure level range into which the rhythm-semantic pressure index falls. The control module is used to encapsulate the script degradation prompt signal and degradation strategy label into auxiliary intervention instructions, generate intervention message packages, and send the auxiliary intervention instructions to the agent terminal interface. The agent terminal interface displays visual script guidance information and outputs real-time call auxiliary analysis results.

[0189] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0190] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A real-time call-assisted analysis method based on ASR, characterized in that, include: Obtain the speech segment sequence segmented by timestamp by the speech recognition engine, calculate the timestamp difference at the boundary of adjacent speech segments, and obtain the pause interval sequence; Based on the pause interval sequence, calculate the sentence-by-sentence change in the pause interval between the current segment and the previous segment to obtain the rhythm fluctuation sequence; extract the fluctuation pattern of the three sentence-by-sentence changes that are continuously arranged along the time axis in the rhythm fluctuation sequence to construct the local pause fluctuation trend. Segments with local pauses and fluctuations that show an initial increase followed by a decrease and a convex shape are classified into the rhythm imbalance zone, while those with smooth fluctuations are classified into the normal expression zone, resulting in a set of labeled segments. The corresponding response speech segments are extracted from the rhythm imbalance area in the labeled segment set and transcribed to obtain the imbalanced response text segment; the word units in the imbalanced response text segment are extracted and matched with the preset high-pressure sensitive word library, and the semantic negative weight is assigned according to the product of the preset sensitivity intensity and the rhythm imbalance amplitude to construct the semantic deviation sequence; the accumulation range is defined segment by segment in the rhythm imbalance area, and the semantic negative weight in the semantic deviation sequence is accumulated sentence by sentence to obtain the rhythm-semantic pressure index; Determine whether the rhythm-semantic pressure index exceeds the preset intervention trigger threshold, and trigger the generation of a speech downgrade prompt signal; match the corresponding downgrade strategy label according to the pressure level range into which the rhythm-semantic pressure index falls; The downgrade prompt signal and downgrade strategy label are encapsulated into auxiliary intervention instructions to generate an intervention message package; the auxiliary intervention instructions are sent to the agent terminal interface, where visual script guidance information is displayed and real-time call auxiliary analysis results are output.

2. The real-time call assistance analysis method based on ASR according to claim 1, characterized in that, The speech segment sequence is obtained by the speech recognition engine and divided by timestamps. The timestamp difference at the boundary of adjacent speech segments is calculated to obtain the pause interval sequence, including: Based on the speech segment sequence, extract the start and end timestamps generated by the speech recognition engine during the processing of each speech segment to obtain a speech segment set containing time boundary information; Based on the set of speech segments containing time boundary information, calculate the difference between the start timestamp of the next speech segment and the end timestamp of the previous speech segment in adjacent speech segments to obtain the silence duration between adjacent speech segments. Based on the silence duration between adjacent segments, the silence durations are arranged sequentially along the continuous call time axis to obtain the pause interval sequence.

3. The real-time call assistance analysis method based on ASR according to claim 2, characterized in that, Based on the pause interval sequence, calculate the sentence-by-sentence change in the pause interval between the current segment and the previous segment to obtain the rhythm fluctuation sequence; extract the fluctuation patterns of the three sentence-by-sentence changes arranged continuously along the time axis in the rhythm fluctuation sequence to construct the local pause fluctuation trend, including: Based on the pause interval sequence, the difference between the pause interval corresponding to the current segment and the pause interval corresponding to the previous segment is calculated to obtain the sentence-by-sentence variation between adjacent segments, revealing the unevenness of breathing rhythm in continuous expression. Based on the sentence-by-sentence changes between adjacent speech segments, the sentence-by-sentence changes are arranged sequentially along the call time axis to obtain a rhythmic fluctuation sequence that reflects the overall speech rate and rhythm fluctuations. Based on the rhythmic fluctuation sequence, three adjacent sentence-by-sentence changes arranged continuously along the time axis are extracted to obtain local rhythmic triplets; The three sentence-by-sentence changes in the local rhythm triplet are mapped to the two-dimensional plane of time-rhythm change, generating three corresponding change nodes. Adjacent change nodes are connected to generate local rhythm polyline segments. Based on the geometric morphological characteristics of the local rhythm polyline segments, the local pause and fluctuation trend is obtained, transforming the one-dimensional numerical comparison into a two-dimensional spatial morphological measurement.

4. The real-time call assistance analysis method based on ASR according to claim 3, characterized in that, The process of obtaining local pauses and fluctuations based on the geometric morphological characteristics of local rhythmic broken line segments includes: Based on the local rhythmic broken line segment, the intermediate change node is extracted, and the vertical deviation distance from the intermediate change node to the line connecting the first and last change nodes is calculated to obtain the local rhythmic arch height, so as to quantify the sudden increase and decrease amplitude of the pause rhythm within the local time window. Based on the closed area enclosed by the local rhythmic broken line segment and the time reference axis, the area of ​​local rhythmic imbalance is calculated to quantify the degree of spatial deviation of abnormal pause rhythm from the overall speech rate stability. Based on the local rhythm broken line segment, extract the span distance along the time axis of the first and last change nodes, calculate the ratio of the local rhythm arch height to the span distance, and obtain the local rhythm sharpness. Based on the local rhythm arch height, local rhythm imbalance area, and local rhythm sharpness, local rhythm broken line segments with local rhythm arch height greater than the preset shape tolerance, local rhythm imbalance area greater than the preset area threshold, and local rhythm sharpness greater than the preset sharpness threshold are marked as first-increase-then-decrease convex shapes, thus obtaining first-increase-then-decrease convex markings to lock in the rhythm imbalance area caused by stress-induced psychological pressure. Based on the local rhythm arch height, the local rhythm broken line segments whose local rhythm arch height is less than or equal to the preset shape tolerance are marked as smooth fluctuation patterns, and smooth fluctuation marks are obtained to confirm the stable expression state of normal thinking. Based on the morphological markers of the increasing and decreasing convex marks and the smooth fluctuation marks, the morphological markers of each local rhythmic segment are sequentially spliced ​​along the time axis to obtain the local pause and fluctuation trend.

5. The real-time call assistance analysis method based on ASR according to claim 4, characterized in that, Segments exhibiting a convex pattern of initial increase followed by decrease, characterized by localized pauses and fluctuations, are categorized into the rhythm imbalance zone, while those with gentle fluctuations are categorized into the normal expression zone, resulting in a set of labeled segments, including: Based on the local pauses and fluctuations, extract the raised markers that increase first and then decrease, and the smooth fluctuation markers contained therein, to obtain a sequence of local morphological markers arranged sequentially along the time axis; Based on the morphological marker results spliced ​​sequentially along the time axis, the change in morphological marker type between adjacent local time windows is detected. When the morphological marker of an adjacent window changes from a smooth undulating marker to a raised marker that first increases and then decreases, the moment of morphological change is recorded to locate the time boundary of the first occurrence of the stress state during the call. Based on the local morphological marker sequence and the morphological transition time, the continuous speech segments corresponding to the local rhythm triplet containing the first increase and then decrease convex markers are selected, and the morphological transition time is used as the starting anchor point to obtain the set of abnormal pause segments. Based on the local morphological marker sequence, the continuous speech segments corresponding to the local rhythm triplet containing the smooth fluctuation marker are selected to obtain the set of smooth pause segments. Based on the set of abnormal pause segments and the set of smooth pause segments, each segment in the set of abnormal pause segments is labeled with a rhythm imbalance zone, and each segment in the set of smooth pause segments is labeled with a normal expression zone, thus obtaining a set of labeled segments.

6. The real-time call assistance analysis method based on ASR according to claim 5, characterized in that, Based on the rhythm imbalance region in the tagged segment set, the corresponding response speech segment is extracted and transcribed to obtain the imbalanced response text segment; the lexical units in the imbalanced response text segment are extracted and matched with a preset high-pressure sensitive word library, and a semantic negative weight is assigned according to the product of the preset sensitivity intensity and the rhythm imbalance amplitude to construct a semantic deviation sequence; within the rhythm imbalance region, the accumulation range is defined segment by segment, and the semantic negative weight in the semantic deviation sequence is accumulated sentence by sentence to obtain the rhythm-semantic pressure index, including: Based on the rhythm imbalance area in the labeled segment set, the corresponding response speech segment is extracted and transcribed into speech recognition to obtain the imbalanced response text segment, and the spatiotemporal mapping relationship between acoustic rhythm features and semantic text content is established. Based on the unbalanced response text segment, extract the internal lexical units to obtain the unbalanced response lexical set; Based on the local pauses and fluctuations, the absolute value of the difference between the middle value and the first and last values ​​of the first and last values ​​of the convex pattern that increases first and then decreases is extracted to obtain the amplitude of rhythm imbalance. Based on the rhythm imbalance amplitude and the imbalance response word set, each word is matched with a preset high-pressure sensitive word library. The semantic negative weight is assigned according to the product of the preset sensitivity intensity and rhythm imbalance amplitude of the corresponding entry in the word library, resulting in a word sequence with negative weight. Based on the word sequence with negative weights, consecutive words belonging to the same rhythm imbalance zone are extracted sequentially along the time axis to obtain the cumulative range of each segment. Based on the cumulative range of segments, the semantic negative weights of the word sequence with negative weights are accumulated sentence by sentence within the range to obtain the rhythm-semantic pressure index.

7. The real-time call assistance analysis method based on ASR according to claim 6, characterized in that, The rhythm-semantic pressure index is obtained by accumulating semantic negative weights in a word sequence with negative weights sentence by sentence within the range, based on the segment-by-segment accumulation range. This includes: Based on the cumulative range of each segment, extract the arrangement number of the current word element to be processed in the rhythm imbalance area to obtain the current word element position index; Based on the current word position index, calculate the number of segment intervals between the current word and the starting boundary of the rhythm imbalance zone to obtain the time distance steps; Based on the time distance steps, according to the preset basic dissipation ratio and the step-incremental attenuation coefficient, the compression dissipation share corresponding to the current word element is calculated to obtain the distance reduction parameter; Based on the distance reduction parameter, the semantic negative weights in the word sequence with negative weights are reduced proportionally to obtain the reduced negative weights of a single sentence. Based on the reduction of negative weights for individual sentences, the rhythm-semantic pressure index is obtained by accumulating the reduction of negative weights for individual sentences from the starting boundary to the current word position within the range of segment accumulation.

8. The real-time call assistance analysis method based on ASR according to claim 7, characterized in that, Determine whether the rhythm-semantic pressure index exceeds the preset intervention trigger threshold, and trigger the generation of a speech degradation prompt signal; Based on the pressure level range into which the rhythm-semantic pressure index falls, match the corresponding degradation strategy label, including: Based on the rhythm-semantic pressure index and the preset intervention trigger threshold, the numerical values ​​of the two are compared to obtain the index crossing status indicator; Based on the index crossing status indicator, when it is confirmed that the trigger line has been crossed, the rhythm-semantic pressure index is extracted and compared with the boundaries of multiple preset pressure level intervals to obtain the current pressure level indicator. Based on the current level of oppression, the corresponding intervention strategy entry is searched in the preset script downgrade strategy library to obtain the downgrade strategy label; Based on the index over-limit status identifier and the downgrade strategy label, the downgrade strategy label is written into the header field of the over-limit prompt message to obtain the message downgrade prompt signal.

9. The real-time call assistance analysis method based on ASR according to claim 8, characterized in that, The downgrade prompt signal and downgrade strategy label are encapsulated into auxiliary intervention instructions to generate an intervention message package; The auxiliary intervention instructions are issued to the agent terminal interface, where visual script guidance information is displayed, and real-time call assistance analysis results are output, including: Based on the script downgrade prompt signal and the downgrade strategy label, the downgrade strategy label is written into the payload field of the script downgrade prompt signal to obtain the encapsulated auxiliary intervention instruction; Based on the encapsulated auxiliary intervention instructions, data is routed and sent according to the preset agent terminal network address to obtain the intervention message packet that reaches the agent terminal interface; Based on the intervention message packet that arrives at the agent terminal interface, the downgrade strategy label is parsed and extracted, and highlighted in the floating prompt area of ​​the agent screen to obtain a visual script guidance interface. Based on the visual script guidance interface, the system receives the subsequent call audio stream after the agent adjusts the script based on the prompts, and obtains the updated real-time call assistance analysis results, thus completing a complete intervention loop from the perception of psychological pressure to the agent's behavioral response.

10. A real-time call assistance analysis system based on ASR, wherein the system implements the method as described in any one of claims 1 to 9, characterized in that, include: The acquisition module is used to acquire the speech segment sequence segmented by the speech recognition engine according to the timestamp, calculate the timestamp difference at the boundary of adjacent speech segments, and obtain the pause interval sequence. The extraction module is used to calculate the sentence-by-sentence change of the pause interval between the current segment and the previous segment based on the pause interval sequence, and obtain the rhythm fluctuation sequence; extract the fluctuation pattern of the three sentence-by-sentence changes arranged continuously along the time axis in the rhythm fluctuation sequence to construct the local pause fluctuation trend. The "Classification" module is used to classify segments with local pauses and fluctuations that show an initial increase followed by a decrease in a convex shape into the rhythm imbalance zone, and segments with smooth fluctuations into the normal expression zone, thus obtaining a set of labeled segments. The transcription module is used to extract the corresponding response speech segments from the rhythm imbalance area in the set of labeled segments and transcribe them to obtain the imbalanced response text segments; extract the word units in the imbalanced response text segments and match them with the preset high-pressure sensitive word library, assign semantic negative weights according to the product of preset sensitivity intensity and rhythm imbalance amplitude, and construct semantic deviation sequences. The calculation module is used to define the cumulative range segment by segment within the rhythm imbalance zone, and accumulate the semantic negative weights in the semantic deviation sequence sentence by sentence to obtain the rhythm-semantic pressure index; determine whether the rhythm-semantic pressure index exceeds the preset intervention trigger threshold, trigger the generation of the speech downgrade prompt signal; and match the corresponding downgrade strategy label according to the pressure level range into which the rhythm-semantic pressure index falls. The control module is used to encapsulate the script degradation prompt signal and degradation strategy label into auxiliary intervention instructions, generate intervention message packages, and send the auxiliary intervention instructions to the agent terminal interface. The agent terminal interface displays visual script guidance information and outputs real-time call auxiliary analysis results.

Images