Real-time call intent recognition method and system

Abstract

The application discloses a real-time call intention recognition method and system, partial stage outputs suspected results quickly through light gate, and satisfies the real-time early warning demand of a seat; final stage independently executes high-confidence final determination based on complete accumulated text, the two are not interfered with each other, the semantic defects and determination lag problems caused by the fragmentation of incremental speech recognition are solved, and the accuracy of the final conclusion is guaranteed under low delay. A large number of invalid partial fragments are filtered through three gate controls of minimum length, trigger word and throttling interval, and the calling amount of a large model is greatly compressed; audio receiving, ASR forwarding and intention analysis are decoupled, semantic reasoning is executed asynchronously and does not block the main link; after callback success, synchronous termination of similar subsequent analysis reduces invalid calculation from the source and improves the concurrent processing throughput of the system. The reasoning adopts an independent bounded timeout background context, and is completely decoupled from the life cycle of a WebSocket connection, so that the loss of the final conclusion caused by the early disconnection of the client is avoided.

Description

Technical Field

[0001] This invention relates to the field of real-time call intent recognition and processing technology, and specifically to a method and apparatus for real-time call intent recognition. Background Technology

[0002] In scenarios such as telephone customer service, sales outbound calls, AICC call centers, and call center quality inspection, systems typically need to identify customer needs from real-time call audio, such as whether the customer wants to add WeChat or WeChat for business, whether they need a quote, whether they want to file a complaint, or whether they need to be transferred to a human operator. Existing solutions usually adopt the following approaches: waiting for the ASR final result before performing keyword matching or model classification; directly performing keyword matching on a single partial segment returned by the ASR; synchronously calling a text classification model or a large model after the ASR final result arrives; and requiring the business system to provide complete business metadata such as callId and streamId before the audio stream begins.

[0003] The above approach has significant technical drawbacks in engineering practice. First, waiting solely for the final ASR result leads to insufficient real-time performance, failing to promptly alert agents or the AICC system when a customer expresses their needs. Second, real-time ASR services like FunASR may output partial text in incremental fragments, such as outputting "add you" first, then "WeChat." If only the latest fragment is used for judgment, a single fragment may not contain complete semantics, easily leading to missed detections. Third, calling the large model for each partial ASR text generates numerous invalid inference requests, increasing costs and end-to-end latency. Fourth, Asterisk ExternalMedia or browser audio streams may quickly close the WebSocket connection after the audio ends. If the final intent analysis is bound to the WebSocket request context, the LLM call may be canceled, resulting in the loss of final confirmation and business callbacks. Therefore, a real-time call intent recognition solution is needed that can utilize partial ASR for real-time alerts and final ASR for final confirmation, while also adapting to different audio protocols, different metadata arrival orders, WebSocket lifecycle changes, and business callback consistency constraints. Summary of the Invention

[0004] The main objective of this invention is to provide a real-time call intent recognition method and system to address the shortcomings of related technologies.

[0005] To achieve the above objectives, according to a first aspect of the present invention, a real-time call intent recognition method is provided, comprising: within a completed initialization session, performing incremental recognition on audio frames of call data based on a real-time speech recognition service to obtain incremental recognition results; for each received incremental recognition result from the real-time speech recognition service, merging and updating the current incremental recognition result with existing accumulated text according to a preset merging rule to obtain the latest accumulated text of the current session, wherein the incremental recognition result includes partial recognition results output segment by segment and final recognition results output at the end of a sentence; when the currently received incremental recognition result is a partial recognition result, based on the... The latest accumulated text triggers a judgment on whether to initiate semantic intent reasoning in the partial stage. If it is determined to be initiated, a monotonically increasing sequence number is assigned to this intent analysis, and a corresponding partial intent analysis task is generated. Semantic intent reasoning is executed asynchronously in an independent thread. Sequence number verification and session stage verification are performed on the asynchronously returned partial intent analysis results. If the verification results meet the specified conditions, a suspected intent event is output. When the received incremental recognition result is a final recognition result, a bounded timeout background context independent of the current audio session connection lifecycle is used to asynchronously execute final intent confirmation reasoning and output a confirmation intent event of the corresponding intent type.

[0006] According to a second aspect of the present invention, a real-time call intent recognition system is provided, comprising an incremental speech recognition text accumulation unit, which, within a completed initial session, performs incremental recognition on audio frames of call data based on a real-time speech recognition service to obtain incremental recognition results; for each received incremental recognition result from the real-time speech recognition service, merges and updates the current incremental recognition result with existing accumulated text according to a preset merging rule to obtain the latest accumulated text of the current session, wherein the incremental recognition results include partial recognition results output segment by segment and final recognition results output at the end of a sentence; and an asynchronous intent recognition unit, which, when the currently received incremental recognition result is a partial recognition result... At that time, based on the latest accumulated text, a trigger judgment is performed to determine whether to initiate semantic intent reasoning in the partial stage; if it is determined to be initiated, a monotonically increasing sequence number is assigned to this intent analysis, and a corresponding partial intent analysis task is generated, which is executed asynchronously in an independent thread for semantic intent reasoning; the sequence number verification and session stage verification are performed on the asynchronously returned partial intent analysis result, and if the verification result meets the specified conditions, a suspected intent event is output; when the received incremental recognition result is a final recognition result, a bounded timeout background context independent of the current audio session connection lifecycle is used to asynchronously execute final intent confirmation reasoning and output a confirmation intent event of the corresponding intent type.

[0007] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing the computer to perform the method described in any one of the first aspects.

[0008] According to a fourth aspect of the present invention, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the method described in any implementation of the first aspect.

[0009] This embodiment of the real-time call intent recognition method and system uses lightweight gating to quickly output suspected results in the partial stage, meeting the real-time alert requirements of agents. In the final stage, a high-confidence final judgment is independently executed based on the complete accumulated text. These two stages operate independently without interference, resolving the semantic incompleteness and judgment lag issues caused by incremental speech recognition fragmentation, and ensuring the accuracy of the final conclusion with low latency. A large number of invalid partial segments are filtered through triple gating using minimum length, trigger words, and throttling intervals, significantly reducing the number of large model calls. Audio reception, ASR forwarding, and intent analysis are decoupled, and semantic inference is executed asynchronously without blocking the main link. Successful callbacks synchronously terminate subsequent analyses of the same type, reducing invalid computations at the source and improving the system's concurrent processing throughput. Inference uses an independent bounded timeout background context, completely decoupled from the WebSocket connection lifecycle, preventing premature client disconnection and loss of the final conclusion. Attached Figure Description

[0010] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0012] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0013] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0014] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0015] According to embodiments of the present invention, a method is provided, such as Figure 1 As shown, steps 101 to 103 are included below: Step 101: Within the initialized session, perform incremental recognition on the audio frames of the call data based on the real-time speech recognition service to obtain incremental recognition results; for each received incremental recognition result from the real-time speech recognition service, merge and update the current incremental recognition result with the existing accumulated text according to the preset merging rules to obtain the latest accumulated text of the current session, wherein the incremental recognition result includes the partial recognition result output by segment refresh and the final recognition result output by sentence end confirmation.

[0016] As an optional implementation of this embodiment, the method for initializing a session includes: accessing upstream transmitted call data via WebSocket, wherein the call data includes service metadata messages and binary audio frames; if the first frame is a message carrying complete service metadata, then extracting the session identifier, audio format parameters, and call role information to complete session initialization; if the first frame is a binary audio frame or does not carry complete service metadata, then generating an internal temporary session identifier to complete session initialization, and marking the session as having metadata pending backfilling; after session initialization is completed, continuously receiving upstream transmitted call data within the initialized session.

[0017] In this step, the session is initialized from scratch, generating a session context capable of carrying all subsequent processing flows. The context stores basic information such as session identifier, audio source format parameters, metadata backfilling status, and call roles, serving as the carrier for all subsequent steps. Within the initialized session, all processing results are specifically attributed to that session.

[0018] For example, audio data is received via WebSocket from Asterisk, a SIP line platform, a browser, or an AICC platform. Session start information can be a standard JSON start message, an audio_stream_started service metadata message, an Asterisk MEDIA_START text control message, or a binary audio frame arriving directly before metadata is received. When the service metadata is complete, the session is initialized using fields such as callId, streamId, speaker, codec, sampleRate, and channels. When the service metadata has not yet arrived, an internal session ID is generated for internal logging and temporary association only, and the callback status is marked as waiting for the real service ID.

[0019] Within the initialized session, for the continuously received binary audio frames, format verification and conversion processing are performed according to the audio format parameters determined during session initialization. After being uniformly converted into a preset standard audio format, the audio is streamed and forwarded to the real-time speech recognition service. The incremental recognition results returned by the real-time speech recognition service are received. The incremental recognition results include partial recognition results that are refreshed segment by segment and final recognition results that are confirmed at the end of the sentence.

[0020] The source format parameters (encoding method, sampling rate, number of channels) used for audio format verification and conversion are all derived from the initialization phase: in scenarios where metadata arrives first, the parameters are extracted from the parsed business metadata; in scenarios where audio arrives first, the parameters use the default adaptation rules set during initialization (e.g., in Asterisk scenarios, ulaw is used by default), and are refreshed and corrected after subsequent metadata arrives. Session initialization is executed only once, occurring at the start of session establishment; it starts immediately after initialization and runs continuously in a loop throughout the entire session, processing arriving audio data frame by frame until the session ends.

[0021] For example, the upstream audio format is verified and necessary conversions are performed. For standard access, the audio is required to be pcm_s16le, 8000 Hz, and mono. For Asterisk ulaw audio, the system decodes the ulaw into 16-bit little-endian PCM and forwards it to the FunASR real-time recognition service. For slin16 or 16000 Hz metadata declared by the browser, the system accepts it as source-end declaration information, but the effective processing contract actually entering ASR remains 8000 Hz mono PCM to avoid ASR input configuration drift.

[0022] Partial results are incremental or intermediate outputs in the speech recognition process, continuously refreshed, supplemented, and corrected as audio input increases; final results mark the end of a single sentence's speech recognition and are the final confirmation output for the entire sentence. They are only generated when ASR detects a pause at the end of the sentence and the speech input for that sentence terminates.

[0023] The preset merging rules are executed as follows: If the currently received incremental recognition result is empty text, the existing accumulated text remains unchanged; if the existing accumulated text is empty, the current incremental recognition result is set as the new accumulated text; if the text content of the current incremental recognition result completely contains the existing accumulated text, the current incremental recognition result completely replaces the existing accumulated text; if the end content of the existing accumulated text completely overlaps with the entire content of the current incremental recognition result, the existing accumulated text remains unchanged; if none of the above conditions are met, the current incremental recognition result is appended to the end of the existing accumulated text to obtain the updated latest accumulated text.

[0024] For example, a two-path processing is performed on each result returned by ASR: on the one hand, the original asr_result is immediately pushed to the WebSocket client to keep the client-visible text consistent with the original ASR result; on the other hand, accumulatedText is maintained in the internal intent recognition chain to trigger word matching and semantic judgment of the large model. The internal accumulated text adopts the following merging rules: if incoming is empty, accumulatedText remains unchanged; if accumulatedText is empty, it is set to incoming; if incoming already contains accumulatedText, it is considered that ASR has returned a full refresh text, and incoming replaces accumulatedText; if accumulatedText ends with incoming, incoming is considered to be a repeated suffix, and accumulatedText remains unchanged; otherwise, incoming is appended to the end of accumulatedText.

[0025] The rule that "partial" always arrives before "final" within a single sentence only applies to the recognition cycle of a single sentence. However, real-time calls involve continuous, multi-turn, multi-sentence interactions, with each sentence independently completing the entire process of "partial incremental refresh → final sentence confirmation." Therefore, throughout the entire call session's lifecycle, the sequence of results returned by ASR (Automatic Speech Recognition) alternates cyclically, for example: Sentence 1: partial1 → partial2 → final1; Sentence 2: partial3 → partial4 → final2; Sentence 3: partial5 → final3 ……

[0026] In other words, after receiving a final result, the next result that arrives is most likely a partial result of the next sentence, rather than continuing to be final. Each incremental recognition result received cannot predict in advance whether it is a partial result of a new sentence or a final result of the previous sentence. Type judgment must be performed first before it is distributed to the corresponding processing logic.

[0027] Step 102: When the currently received incremental recognition result is a partial recognition result, a trigger judgment is performed based on the latest accumulated text to determine whether to initiate semantic intent reasoning in the partial stage; if it is determined to be initiated, a monotonically increasing sequence number is assigned to this intent analysis, and a corresponding partial intent analysis task is generated to perform semantic intent reasoning in an asynchronous independent thread; the sequence number verification and session stage verification are performed on the asynchronously returned partial intent analysis result, and if the verification result meets the specified conditions, a suspected intent event is output.

[0028] As an optional approach in this embodiment, the Xi'an method involves determining whether to initiate partial semantic intent reasoning based on the latest accumulated text. This includes: determining whether the character length of the latest accumulated text is not less than a preset minimum text length threshold; determining whether the latest accumulated text contains a preset trigger word corresponding to the target intent type; determining whether the interval between the current time and the time of the last partial semantic intent reasoning initiation in the current session is not less than a preset minimum request interval threshold; and initiating partial semantic intent reasoning if all conditions are met.

[0029] In this optional implementation, for partial ASR results, instead of directly calling LLM on all fragments, a lightweight trigger judgment is first performed based on accumulatedText. Trigger configuration includes whether partial triggering is enabled, the trigger word list, the minimum text length, and the minimum interval for partial LLM requests within the same call. Trigger words can include WeChat, Enterprise WeChat, QR code, add you, you add me, add WeChat, send code, etc. Only when accumulatedText reaches the minimum length, contains the trigger word, and meets the throttling interval, is LLM asynchronously called to perform suspected intent judgment.

[0030] As an optional implementation in this embodiment, the asynchronously returned partial intent analysis results are subjected to sequence number verification and session phase verification. If the verification results meet the specified conditions, the suspected intent event is output, including: recording the latest sequence number corresponding to all partial intent analysis tasks initiated in the current session, wherein the sequence number monotonically increases with the initiation of the task; after receiving the partial intent analysis result, the sequence number corresponding to the result is first compared with the current latest sequence number, and then it is determined whether the current session has entered the final processing stage; the corresponding suspected intent event is allowed to be output only when the sequence number corresponding to the result is consistent with the current latest sequence number and the current session has not yet entered the final processing stage, wherein partial intent analysis results whose sequence number is behind the latest number or whose session has entered the final processing stage are directly discarded.

[0031] The semantic intent reasoning in the partial phase is executed asynchronously and independently, without blocking the ASR result reception process. Due to the unstable time consumption of large model reasoning, the order of returned reasoning results may differ from the order initiating requests. Typical scenarios include: partial reasoning requests initiated earlier, due to their longer reasoning time, return results later than those initiated later; even after the final recognition result has arrived and the session has entered the final processing phase, earlier partial reasoning results arrive late. Therefore, a sequence number verification and final phase priority mechanism is designed. Each partial reasoning request is assigned a monotonically increasing sequence number. When a result is returned, it is first verified to be the latest sequence number; results with older sequence numbers are discarded without overwriting updated context judgments. Once a final recognition result is received, the session is immediately marked as entering the final processing phase. Afterward, regardless of whether partial reasoning results are returned, no more suspected intent events are output, ensuring that the priority of the final conclusion is not affected by asynchronous out-of-order processing.

[0032] For example, partial intent results are asynchronously sorted and old ones are removed. Each partial LLM request is assigned a monotonically increasing sequence number. LLM calls are executed in independent asynchronous tasks, without blocking ASR forwarding and audio reception. The specific implementation is achieved by decoupling three links: audio reception, ASR result forwarding, and intent analysis. 1. The audio receiving link operates independently. The WebSocket main loop continuously executes conn.ReadMessage(), and upon receiving binary audio, immediately performs the necessary ulaw to PCM conversion and calls stream.SendAudio() to forward it to the ASR service. This link does not directly call LLM, nor does it wait for intent analysis results.

[0033] 2. The ASR result forwarding chain operates independently. After starting an ASR session, the system launches a separate goroutine via `goforwardASRResults(...)` to read the ASR return results. Upon receiving each ASRResult, it immediately pushes the raw `asr_result` to the client using `writeJSON`, and then calls `runner.handle(...)` for internal intent processing. Therefore, ASR result forwarding takes precedence over LLM judgment.

[0034] 3. The `runner.handle(...)` function performs only lightweight synchronous processing: updating `accumulatedText`, recording the history, checking for `final` / `partial`, and determining trigger words and throttling conditions. Once the `partial` trigger condition is met, it only increments `partialSeq`, records `lastPartialAt`, and then starts asynchronous LLM analysis via `go r.runPartial(...)` and returns immediately. In other words, the actual time-consuming HTTP LLM requests are not executed within the ASR result reading goroutine.

[0035] 4. Partial LLM results are prevented from being out of order using sequence numbers. Each partial request has a monotonically increasing sequence number (seq). After the LLM returns, only results that meet the conditions of "not yet entering the final state, not yet being suspected, and having a sequence number equal to the latest partial sequence number" are allowed to be emitted. This ensures that even if an older request returns later, it will not overwrite the new context or block the processing of new ASR fragments.

[0036] 5. The final LLM also executes asynchronously. After the final ASR arrives, `runner.handle(...)` sets `finalReached`, and `runFinal(...)` is called via a goroutine. `runFinal` uses `context.WithTimeout(context.Background(), 5s)`, no longer relying on the WebSocket request context. Therefore, even if the upstream audio connection is closed, the final analysis can still complete within the time limit, ensuring that the AICC callback chain has a chance to continue executing.

[0037] 6. WebSocket writes use `writeJSON` for serialization, protecting only the threads that write to the socket concurrently. Since ASR results, `suspected`, and `confirmed` might all be written to the same connection, `writeJSON` uses a mutex lock to prevent concurrent write conflicts. This lock only constrains the order of "write client events" and does not participate in audio reading and `SendAudio`, thus not blocking the audio reception link.

[0038] When multiple partial requests return concurrently, only the latest result with the latest sequence number that has not yet entered the final stage is allowed to issue a suspected intent event; older results or partial results returned after the final stage are ignored. A single call will issue a suspected intent event at most once to avoid interference from multiple real-time alerts on the front-end or business systems.

[0039] Furthermore, the intent analysis result in this embodiment is the raw judgment conclusion returned by the semantic intent reasoning service (large model / text classification model), which is intermediate processing data and is not directly output. The core content includes a binary classification conclusion of whether the target intent is hit, confidence level, and matching text evidence, such as outputting only a "1" or "0" result. Each intent analysis result clearly corresponds to a preset intent type, such as add_wechat (add WeChat), quote request, complaint intent, or transfer to human agent intent, which is a specific judgment conclusion for that type of need. Sequence number verification and session phase verification are performed on the intent analysis results. After successful verification, the judgment conclusion of the analysis result, the corresponding intent type, along with session context, text evidence, and other information, are encapsulated to generate the corresponding intent event.

[0040] Step 103: When the received incremental recognition result is a final recognition result, a bounded timeout background context independent of the current audio session connection lifecycle is used to asynchronously execute the final intent confirmation reasoning and output the corresponding intent type confirmation intent event.

[0041] The "suspected alert + final confirmation" two-stage approach essentially balances the conflicting goals of real-time performance and accuracy. If only the final recognition result is waited for before intent determination, a conclusion can only be output after the customer has fully expressed their needs, preventing the agent or AICC system from responding as soon as the customer mentions their request. For example, when a customer says "add you on WeChat," a suspected result can trigger a pop-up alert on the agent's end, without waiting for the entire sentence to finish or for the final result to be generated. This is a core requirement for scenarios such as agent assistance and real-time script recommendation, providing timely alerts to agents or the AICC system as soon as the customer expresses their needs. Performing complete semantic reasoning on every partial fragment would generate numerous invalid calls and increase computational costs. The suspected stage uses lightweight gating with "trigger words + minimum length + throttling intervals" to filter first, initiating reasoning only when there is a clear intent tendency. This trades off real-time benefits for extremely low rule costs, controlling the cost of large model calls and reducing invalid reasoning requests.

[0042] The two are designed for completely different use cases. Suspected intent events are for the front-end real-time interaction link (agent terminal, AICC real-time dialogue system), and their role is to provide advance reminders, allowing for a certain margin of error, with the priority of "speed". Confirmed intent events (final conclusions) are for the back-end business database link (CRM, quality inspection, business statistics system), and their role is to make formal judgments and retention, with the priority of "accuracy".

[0043] For example, when the ASR returns a final result, the final text is first merged into the accumulatedText, and then a confirmed intent analysis is always performed once. This final analysis uses a bounded timeout context independent of the WebSocket request context. Even if the client quickly closes the connection after sending end or after the audio finishes, the final intent analysis can still complete within the timeout period. If the final LLM is determined to be a hit, a confirmed intent result is generated, and the business callback process continues. If the WebSocket has been closed, real-time push may fail, but the callback chain can still continue to execute.

[0044] Specifically, when the currently received incremental recognition result is a final recognition result, after merging the final recognition result into the accumulated text according to the preset merging rules and completing the final update, a background execution context with a preset timeout duration is created, independent of the current audio session WebSocket connection lifecycle. In the background execution context, the final semantic intent confirmation reasoning is always performed only once. If the reasoning result is a hit with the corresponding intent, a confirmation intent event of the corresponding intent type is generated and output. Even if the WebSocket connection is closed, causing real-time push to fail, the confirmation intent event still enters the subsequent business callback process normally.

[0045] The `final` analysis uses a bounded timeout context independent of the WebSocket request context. The core of this design is to decouple the inference task from the WebSocket connection's lifecycle: In conventional solutions, the inference task is nested within the WebSocket request processing coroutine / thread; a connection break leads to context destruction and task termination. This solution separates the `final` inference task from the main WebSocket chain and places it in an independent background execution environment, controlling the maximum execution time only through a preset timeout. This fundamentally solves the problem of lost final confirmation due to rapid connection closure. For a single `final` recognition result, the final confirmation inference is executed exactly once, unaffected by the number of partial stage triggers or hit results, ensuring the independence and authority of the `final` conclusion.

[0046] Specifically, the background execution context with a preset timeout duration includes an independent asynchronous execution unit, a bounded timeout control mechanism, a full input data copy, and an independent result output link. The final semantic intent confirmation reasoning operation is as follows: the final complete cumulative text obtained after merging the final recognition results is used as the only input. All data required for reasoning is generated into a local copy and passed to the background execution context, without performing a separate secondary text merging. The reasoning task is carried out by an independent asynchronous execution unit and is not attached to the request processing link corresponding to the WebSocket connection. Closing the WebSocket connection will not interrupt or cancel the reasoning task. The maximum runtime of the task is controlled by the bounded timeout control mechanism. If the task is completed within the preset time, the result is output normally. If the preset time exceeds the time, it will automatically terminate. After the reasoning is completed, the confirmation intent event is directly written to the business callback queue through the independent result output link. Even if the WebSocket connection is closed, causing real-time push to fail, the confirmation intent event can still enter the subsequent business callback process normally. Regardless of whether semantic intent reasoning was initiated or suspected intent events were output in the partial stage, the final stage executes the final semantic intent confirmation reasoning once, and the reasoning conclusion is not affected by the result of the partial stage.

[0047] As an optional implementation of this embodiment, the method further includes: uniformly judging the metadata status of the current session for suspected intent events and confirmed intent events; if the session is in the metadata pending backfilling state, then the intent event is marked as pending callback state and stored in the pending callback queue, without immediately initiating a business callback to the upstream business system; when the session receives the business metadata containing the real business identifier sent by the upstream, it uses the real business identifier to backfill all intent events in the pending callback queue, regenerates the legal event identifier, and then enters the business callback process; if the real business identifier is not obtained when the session ends, then all intent events in the pending callback queue are discarded.

[0048] In this optional implementation, the arrival order of business metadata is unstable. Binary audio or MEDIA_START may arrive before the actual callId of AICC. If a temporary channel ID callback is used directly, it will pollute the business system. If ASR is started after waiting for the metadata, the front-end audio will be lost or the processing delay will be increased.

[0049] Business callback is a standardized HTTP notification mechanism that delivers intent recognition results to upstream business systems (AICC call center, CRM, agent support system, etc.): After determining a valid customer intent, the intent event is proactively pushed to the designated interface of the business system according to the agreed format, triggering subsequent processing on the business side (such as customer demand marking, agent pop-up reminders, quality inspection event archiving, etc.).

[0050] The true business identifier originates entirely from business metadata messages sent by the upstream business system through the same WebSocket connection, such as the standard JSON start message and the audio_stream_started business metadata message. The core identifier is the genuine AICC callId (unique business ID for the call), accompanied by related business fields such as streamId (audio stream identifier), speaker (call role), and business scenario. It is a formal business identity recognizable by the upstream business system, distinct from self-generated internal temporary session identifiers used only for temporary internal associations. It can arrive before or after the binary audio frame; the metadata pending backfill status is the intermediate state of waiting for this true business identifier. The backfilling action essentially replaces the internal temporary identifier with the true business identifier sent by the upstream system, ensuring that all fields in the external callback are formal content recognizable by the business system, preventing internal temporary IDs from polluting the business system.

[0051] Metadata backfilling and callback admission are the pre-gates for callbacks, and the core solution is to address the issue of whether the callback identity is legitimate: all intent events (suspected, confirmed) must be bound to a real business identifier before they can be sent out; if there is no real identifier, they are temporarily stored in a pending state, and the event is allowed after the identifier arrives; if there is no valid identifier throughout the process, the event is directly discarded, ensuring from the source that the business system will not receive invalid internal temporary ID events.

[0052] Whether it's the real-time alert (suspected intent event) output in the partial stage or the final conclusion (confirmed intent event) output in the final stage, as long as a formal callback needs to be sent, it must first undergo verification to see if it holds the real business ID. It cannot be sent out directly after the result is generated. Therefore, for both suspected intent events and confirmed intent events, the metadata status of the current session is uniformly judged.

[0053] Furthermore, as mentioned above, the metadata pending state is a state defined during initialization: the current session only has a self-generated internal temporary session identifier and has not yet obtained formal identity information such as the real callId recognized by the upstream business. If a callback is sent directly at this time, the internal temporary ID cannot be recognized, resulting in dirty data. Therefore, the generated intent event is first stored in the "pending callback queue" (pending state), which is only kept internally and not sent out.

[0054] Once the real business identifier (real callId, streamId, call role, etc.) arrives, first update the identity information of the session itself, and then process all temporarily stored intent events in the queue in batches: replace all internal temporary identifiers associated with the events with the real business identifiers; clear the event IDs (eventId) originally generated based on the internal temporary IDs, and regenerate event IDs that conform to the business system specifications based on the real business identifiers, intent types, timestamps, and other information.

[0055] Only after backfilling is completed does the event gain the legitimacy of a formal callback and enter the subsequent business callback process. If no formal business metadata is received by the end of the entire call, all temporarily stored intent events are discarded directly, and no callbacks carrying internal temporary IDs are sent to the business system throughout the entire process, thus completely preventing internal identifiers from polluting the external business system.

[0056] For example, if a customer request such as "add_wechat" is detected before the actual AICC callId is obtained, the system does not immediately send a business callback but instead saves the intent result as a pending state. Once the business metadata arrives, the system updates the session metadata with the actual callId, streamId, and speaker, and refreshes the pending intent results in the order of arrival. When refreshing the pending results, the original eventId based on the internal temporary ID is cleared, and a new eventId is generated based on the actual callId, intent type, state, timestamp, streamId, and incrementing sequence number, ensuring that external business systems do not receive the internal channel ID. If the actual callId has not arrived by the end of the session, the pending callback is discarded, only the log is retained, and no temporary ID callback is sent to the business system.

[0057] As an optional implementation of this embodiment, the method further includes: for all intent events entering the business callback process, performing deduplication control using the combination of the real business identifier and the intent type as the unique idempotent key; under the same call and the same intent type, only one successful business callback is allowed; when a business callback of a certain intent type is successful, the call and that type of intent are marked as notified, and all subsequent analysis tasks and callback attempts of the same type of intent are stopped.

[0058] In this optional implementation, the suspected and confirmed multi-stage results may hit the same customer requirement. If idempotency and suppression mechanisms are lacking, the business system may receive duplicate events.

[0059] All intent events (including suspected and confirmed events) that pass metadata verification and possess legitimate business identities will uniformly use a combination string of the real business identifier and intent type as the sole basis for deduplication (i.e., an idempotent key). For example, the real business identifier, such as the AICC real callId, uniquely identifies a call; the intent type, such as add_wechat, complaint, or transfer to a human agent, uniquely identifies a type of customer need; the combination of the two can uniquely pinpoint a specific type of customer need within a specific call, serving as the smallest granularity for deduplication.

[0060] A unified deduplication standard ensures that the same requirement in the same call will not repeatedly send notifications to the business system due to multiple partial inferences, multiple ASR fragment hits, or suspected + confirmed two-stage hits.

[0061] Within a single intent dimension for a single call, regardless of the number of intent events generated sequentially (suspected alerts, multiple partial hits, final confirmations, etc.), only one successful business callback will ultimately be delivered to the business system. If a suspected intent event successfully callbacks first, subsequent confirmed intent events of the same type will be directly suppressed and will not be resent; if a suspected callback fails, the confirmed intent event can still attempt to callback. This ensures the uniqueness of business-side events and avoids repeatedly triggering business processes such as customer request follow-up and agent pop-ups, which could cause confusion in business logic.

[0062] When a business callback of a certain intent type succeeds, the call and that intent type are marked as notified, stopping all subsequent analysis tasks and callback attempts for the same intent. Status Marking: Immediately after a successful callback, the intent is marked as notified within the current session, serving as the basis for deduplication of all subsequent similar events. Source Termination: Not only are subsequent callbacks for similar events stopped, but the initiation of subsequent analysis tasks for the same intent is also directly stopped (including partial trigger judgments, LLM inference calls, etc.), terminating invalid computations at the source. This approach not only "prevents duplicate notifications" but also directly terminates subsequent useless semantic analysis, further reducing the cost of large model calls and improving processing efficiency.

[0063] For example, only intent results that meet the conditions are sent via business callbacks, such as the result of `wechat_contact` with `decision=true`. The callback payload is mapped to AICC customer request events, such as `needs[].type=add_wechat` or `needs[].label=Customer wants to add WeChat`. The system uses `callId + add_wechat` as the idempotent key to suppress duplicate callbacks within the same call. Network errors, timeouts, and 5xx responses can be retried using the same `eventId`; 4xx responses are not retried. If a callback succeeds, the call is marked as having sent the request, and subsequent similar intent analysis or callbacks are stopped to avoid duplicate notifications caused by `suspected` and `confirmed` or multiple ASR fragments.

[0064] The application scenarios of the method in this embodiment are as follows: The following example illustrates how to identify a customer's request to "add WeChat or WeChat for Business" in a sales scenario.

[0065] 1. The upstream system receives call audio via WebSocket. If the first frame is a start message, it contains fields such as call_id, scenario=sales, format=pcm_s16le, sample_rate=8000, channels=1, stream_id, and speaker=customer. If the first frame is an Asterisk MEDIA_START or binary audio, the system first starts ASR with an internal temporary ID and waits for subsequent business metadata.

[0066] 2. The system forwards the binary audio frame to FunASR. If the input is ulaw, it is first decoded to PCM. FunASR may return the partial text "Add you", the partial text "WeChat", and the final text "The customer said I added you on WeChat" in sequence.

[0067] 3. The system pushes each asr_result externally as is, but internally merges "Add You" and "WeChat" into "Add You on WeChat". When accumulatedText matches "WeChat" or "Add You on WeChat" in partial_trigger.terms, and more than interval_ms has passed since the last partial LLM call, the system asynchronously initiates a suspected check.

[0068] 4. LLM prompts require the model to output only 1 or 0. If accumulatedText indicates an intention to exchange WeChat or Enterprise WeChat contact information during the current call, output 1; if it is just WeChat Pay, Official Account, Moments, or history reposts, output 0.

[0069] 5. If partial LLM returns 1, the system outputs a suspected intent event with fields including type=intent_result, intent=wechat_contact, status=suspected, decision=true, evidence=add you on WeChat, and is_final=false.

[0070] 6. When the final ASR arrives, the system always calls LLM again based on the final accumulatedText. This call uses a timeout context independent of the WebSocket lifecycle. If it returns 1, the confirmed intent event is output.

[0071] 7. If the actual AICC callId is known, the system will convert the confirmed or suspected hit result into an AICC callback. The payload includes callId, streamId, eventId, eventType=customer_need_detected, speaker, needs[].type=add_wechat, needs[].label=customer wants to add WeChat, confidence, and evidence.

[0072] 8. If the actual callId has not yet arrived, the system will temporarily store the above intent result. After the audio_stream_started or start metadata arrives, the callId and streamId will be updated, and the eventId will be regenerated before the callback is sent.

[0073] 9. Under the same callId, the system uses callId:add_wechat as the key for deduplication. If the suspected callback has been successfully executed, the confirmed callback will not be repeated; if the suspected callback has failed, the confirmed callback can continue to attempt to execute.

[0074] It should be understood that the ASR service can be replaced with other real-time speech recognition engines, and the output can be partial / final, interim / final, or similar streaming recognition results. The semantic analysis model can be a large intranet model, a lightweight text classification model, or a combination of rule-based and large models, and the output format can be binary classification, intent labels, or structured JSON. Trigger word list, minimum text length, throttling time, final timeout, and deduplication TTL can all be configured according to the business scenario. The business callback object is not limited to AICC; it can also be a CRM, ticketing system, agent assistance system, or quality inspection system. Customer request types are not limited to add_wechat; they can also be extended to quote requests, complaint intents, transfer to human agent intents, after-sales consultations, etc. The audio protocol is not limited to Asterisk or browser WebSocket; it can also be adapted to SIP line platforms, media gateways, or other real-time audio access protocols. Compared with conventional solutions, the method in this embodiment has at least the following technical advantages. First, by building accumulated text from ASR partial fragments, it solves the semantic incompleteness problem caused by fragmentation such as "add you" and "WeChat," reducing the risk of missed detections. Second, by controlling LLM calls through trigger words, minimum text length, and throttling intervals, semantic analysis is initiated only when there is potential customer demand, reducing the number of invalid inferences and system costs. Third, through a two-stage mechanism of partial suspected and final confirmed, it retains the ability to make final confirmations while ensuring real-time alerts. Fourth, by using a bounded context independent of the WebSocket lifecycle for final analysis, final confirmation and business callbacks can still be completed after the connection is closed. Fifth, by using a pending queue, real callId backfilling, and eventId reconstruction, it is compatible with the protocol order where audio arrives first and business metadata arrives later, avoiding temporary IDs polluting the business system. Sixth, by using callId + needType idempotent deduplication and stopping similar analyses after success, it avoids duplicate business events caused by suspected and confirmed or multiple ASR fragments.

[0075] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0076] According to an embodiment of the present invention, a real-time call intent recognition system is also provided, including an incremental speech recognition text accumulation unit. Within a completed initial session, the system performs incremental recognition on audio frames of call data based on a real-time speech recognition service to obtain incremental recognition results. For each received incremental recognition result from the real-time speech recognition service, the system merges and updates the current incremental recognition result with existing accumulated text according to a preset merging rule to obtain the latest accumulated text for the current session. The incremental recognition results include partial recognition results output segment by segment and final recognition results output at the end of a sentence. An asynchronous intent recognition unit is also provided. When the currently received incremental recognition result is a partial recognition result... If the result is obtained, a trigger judgment is performed based on the latest accumulated text to determine whether to initiate semantic intent reasoning in the partial stage. If it is determined to be initiated, a monotonically increasing sequence number is assigned to this intent analysis, and a corresponding partial intent analysis task is generated. Semantic intent reasoning is executed asynchronously in an independent thread. Sequence number verification and session stage verification are performed on the asynchronously returned partial intent analysis result. If the verification result meets the specified conditions, a suspected intent event is output. When the received incremental recognition result is a final recognition result, a bounded timeout background context independent of the current audio session connection lifecycle is used to asynchronously execute final intent confirmation reasoning and output a confirmation intent event of the corresponding intent type.

[0077] According to embodiments of the present invention, the present invention also provides an electronic device, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the methods described in any of the above embodiments.

[0078] According to embodiments of the present invention, the present invention also provides a readable storage medium storing computer instructions that enable a computer to perform the methods described in any of the above embodiments when executed.

[0079] According to embodiments of the present invention, the present invention also provides a computer program product that, when executed by a processor, can implement the methods described in any of the above embodiments.

[0080] Figure 2A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices.

[0081] like Figure 2 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0082] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0083] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the object matching method. For example, in some embodiments, the object matching method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the methods described above may be performed.

[0084] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0085] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0086] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

Claims

1. A method for real-time call intent recognition, characterized in that, include: Within the initialization session, incremental recognition is performed on the audio frames of the call data based on the real-time speech recognition service to obtain the incremental recognition result; For each incremental recognition result received from the real-time speech recognition service, the current incremental recognition result is merged and updated with the existing accumulated text according to a preset merging rule to obtain the latest accumulated text of the current session. The incremental recognition result includes the partial recognition result output by refreshing segment by segment and the final recognition result output by confirming the end of the sentence. When the currently received incremental recognition result is a partial recognition result, a trigger judgment is performed based on the latest accumulated text to determine whether to initiate semantic intent reasoning in the partial stage; if it is determined to initiate, a monotonically increasing sequence number is assigned to this intent analysis, and a corresponding partial intent analysis task is generated, which is executed asynchronously in an independent thread for semantic intent reasoning; the sequence number verification and session stage verification are performed on the asynchronously returned partial intent analysis result, and if the verification result meets the specified conditions, a suspected intent event is output; When the received incremental recognition result is a final recognition result, a bounded timeout background context independent of the current audio session connection lifecycle is used to asynchronously execute the final intent confirmation reasoning and output the corresponding intent type confirmation intent event.

2. The real-time call intent recognition method according to claim 1, characterized in that, The triggering judgment based on the latest accumulated text for semantic intent reasoning regarding whether to initiate the partial phase includes: Determine whether the character length of the latest accumulated text is not less than a preset minimum text length threshold; Determine whether the latest accumulated text contains a preset trigger word corresponding to the target intent type; Determine whether the interval between the current time and the time of the last partial semantic intent inference in the current session is not less than the preset minimum request interval threshold. If both conditions are met, semantic intent reasoning in the partial phase is initiated.

3. The real-time call intent recognition method according to claim 1, characterized in that, Perform sequence number validation and session phase validation on the asynchronously returned partial intent analysis results. If the validation results meet the specified conditions, output the suspected intent events, including: Record the latest sequence number of all partial intent analysis tasks initiated in the current session, where the sequence number monotonically increases as the task is initiated; After receiving the partial intent analysis result, first compare the sequence number corresponding to the result with the latest sequence number, and then determine whether the current session has entered the final processing stage; The corresponding suspected intent event is allowed to be output only if the sequence number corresponding to the result is consistent with the latest sequence number and the current session has not yet entered the final processing stage. Partial intent analysis results with sequence numbers that are later than the latest number or the session has entered the final processing stage are directly discarded.

4. The real-time call intent recognition method according to claim 1, characterized in that, When initializing a session, the methods include: The call data transmitted upstream is accessed via WebSocket, wherein the call data includes business metadata messages and binary audio frames; If the first frame is a message carrying complete business metadata, then extract the session identifier, audio format parameters, and call role information to complete the session initialization. If the first frame is a binary audio frame or does not carry complete business metadata, an internal temporary session identifier is generated to complete session initialization, and the session is marked as having metadata pending backfilling. Once the session is initialized, it continues to receive call data transmitted from upstream within the initialized session.

5. The real-time call intent recognition method according to claim 1, characterized in that, The method also includes: For suspected intent events and confirmed intent events, the metadata status of the current session is uniformly determined; if the session is in the state of metadata pending filling, the intent event is marked as pending callback and stored in the pending callback queue, without immediately initiating a business callback to the upstream business system. When the session receives business metadata containing the real business identifier from the upstream, it uses the real business identifier to populate all intent events in the callback queue, regenerates the valid event identifier, and then enters the business callback process. If the real business identifier is not obtained by the end of the session, all intent events in the callback queue are discarded.

6. The real-time call intent recognition method according to claim 5, characterized in that, The method also includes: For all intent events entering the business callback process, deduplication control is performed using the combination of the real business identifier and intent type as the unique idempotent key; under the same call and the same intent type, only one successful business callback is allowed; when a business callback of a certain intent type is successful, the call and that type of intent are marked as notified, and all subsequent analysis tasks and callback attempts of the same type are stopped.

7. The real-time call intent recognition method according to claim 6, characterized in that, Using the combination of the real business identifier and intent type as the unique idempotent key for deduplication control includes: If a suspected intent event has already been successfully completed and a business callback has been initiated for the same call and intent type, subsequent confirmation intent events of the same type will no longer initiate business callbacks. When a business callback fails, a limited number of retries are made using the same event identifier for server-side error and network timeout responses, but no retries are made for client-side error responses. Once a successful response is received for a business callback of a certain intent type, the call is marked as notified for that intent type, and all subsequent analysis tasks and callback attempts for the same intent type are stopped.

8. A real-time call intent recognition system, characterized in that, include: The incremental speech recognition text accumulation unit performs incremental recognition on the audio frames of the call data based on the real-time speech recognition service within the initialized session to obtain the incremental recognition result. For each incremental recognition result received from the real-time speech recognition service, the current incremental recognition result is merged and updated with the existing accumulated text according to a preset merging rule to obtain the latest accumulated text of the current session. The incremental recognition result includes the partial recognition result output by refreshing segment by segment and the final recognition result output by confirming the end of the sentence. The asynchronous intent recognition unit, when the currently received incremental recognition result is a partial recognition result, performs a trigger judgment based on the latest accumulated text to determine whether to initiate semantic intent reasoning in the partial stage; if the judgment is to initiate, it assigns a monotonically increasing sequence number to this intent analysis, generates a corresponding partial intent analysis task, and executes semantic intent reasoning in an asynchronous independent thread; it performs sequence number verification and session stage verification on the asynchronously returned partial intent analysis result, and outputs a suspected intent event if the verification result meets the specified conditions; when the received incremental recognition result is a final recognition result, it uses a bounded timeout background context independent of the current audio session connection lifecycle to asynchronously execute final intent confirmation reasoning and outputs a confirmation intent event of the corresponding intent type.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method according to any one of claims 1-7.

10. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the method according to any one of claims 1-7.

Images