Voice instruction scheduling method and device, electronic device, and program product

Abstract

The application discloses a voice instruction scheduling method and device, electronic equipment and program product, and relates to the technical fields of artificial intelligence and man-machine interaction, wherein the method comprises: acquiring an audio stream and a current locking state; detecting the audio stream to obtain a voice segment, and determining a target routing channel based on a preset routing function; identifying the voice segment based on the target routing channel to obtain an intent category, creating a voice instruction task based on the current locking state and the intent category; determining the invalidity of each historical voice instruction task to obtain a determination result, and replacing the historical voice instruction task with the voice instruction task based on the determination result; and scheduling the voice instruction task based on a scheduling priority. The application solves the technical problem that concurrent control and cooperation between voice instructions and business operations cannot be realized in related technologies, so that voice instructions cannot be accurately scheduled for voice task processing according to the state of the current business operation.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and human-computer interaction technology, and more specifically, to a method and apparatus for scheduling voice commands, an electronic device, and a program product. Background Technology

[0002] With the development of artificial intelligence technology, voice interaction has been gradually introduced into various self-service terminals to improve user experience and service efficiency. However, the self-service terminals that have introduced voice interaction currently face the following problems in application: (1) Coupling conflict between voice interaction and business operations. The existing self-service terminals usually adopt a simple serial processing mode for voice interaction. When responding to voice commands, business operations will be blocked, or voice input will be completely ignored when performing business operations. In scenarios involving multi-step business processes (such as baggage check-in confirmation, identity verification, payment operations, etc.), there is a lack of effective coordination mechanism between the user's voice input and the ongoing business operations, which can easily lead to the unexpected interruption of business processes or the incorrect overwriting of key operations. (2) Concurrent competition of voice commands. Passengers may issue multiple voice commands in a short period of time, or issue new commands before the processing of the previous command is completed. There is a lack of effective management mechanism for multiple concurrent voice commands, which may lead to confusion in command processing, repeated execution, or omission of execution. (3) Security guarantee of key business operations. Self-service terminals involve key business operations that affect the rights and interests of passengers, such as baggage check-in confirmation (involving the transfer of baggage security responsibility), ticket rescheduling (involving fee changes), identity information verification, etc. These operations cannot be easily undone once executed, but existing self-service terminals have failed to establish an effective voice command protection mechanism for these critical operations, which poses a security risk. (4) Global management of voice session status: Current self-service terminals usually treat each voice input as an independent event and lack global status management of the entire interactive session. When the business process involves multiple steps and multiple confirmations, it is difficult to distinguish the user's new intentions and confirmation responses to the current process, and it is also difficult to correctly restore the session status after the user leaves midway.

[0003] Due to the aforementioned issues, in the voice interaction scenario of self-service terminals, existing technologies cannot achieve concurrent control and collaborative management between voice commands and business operations, resulting in the inability to accurately and intelligently schedule voice commands for voice command task processing based on the current business operation status.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and program product for scheduling voice commands, to at least solve the technical problem in related technologies where concurrent control and coordination between voice commands and business operations cannot be achieved, resulting in the inability to accurately schedule voice commands for voice task processing based on the current state of business operations.

[0006] According to one aspect of the embodiments of this application, a method for scheduling voice commands is provided, comprising: acquiring an audio stream of a passenger monitored by a self-service baggage check-in device, and acquiring the current locking state of the business operation interface of the self-service baggage check-in device; performing voice activity detection on the audio stream to obtain voice segments, and routing the voice segments based on a preset routing function to determine a target routing channel, wherein the preset routing function is used to control the voice segments to enter the target routing channel based on the current locking state; recognizing the voice segments based on the target routing channel to obtain an intent category, and creating a voice command task based on the current locking state and the intent category, wherein... Each voice command task has a corresponding task status. Each historical voice command task in the task queue to be scheduled is assessed for failure, and based on the assessment result, the historical voice command task is replaced with a new voice command task. The task queue to be scheduled stores a set of tasks selected from the task registry. This set includes voice command tasks in the created and queued states. The task registry stores multiple created historical voice command tasks. The scheduling priority of each historical voice command task in the task queue is determined, and the voice command task is scheduled based on the scheduling priorities of all historical voice command tasks.

[0007] Furthermore, before obtaining the current locked state of the business operation interface of the self-service baggage check-in device, the process includes: constructing a finite state automaton, wherein the finite state automaton includes: a locked state set, an event set, a state transition function, an initial locked state, and a mapping function. The locked state set includes: an idle state, a soft locked state, and a hard locked state. The event set includes: multiple business events triggered by the passenger performing business operations on the self-service baggage check-in device. The state transition function is used to transition the current locked state based on the business events in the event set.

[0008] Furthermore, after obtaining the current lock status of the business operation interface of the self-service baggage check-in device, the process includes: if the current lock status of the self-service baggage check-in device is a hard lock status, determining the duration of the current lock status being in the hard lock status; if the duration exceeds the preset allowed duration, converting the hard lock status to a soft lock status and rolling back any unfinished business events.

[0009] Further, the step of identifying speech segments and obtaining intent categories based on the target routing channel includes: when the target routing channel is an intent recognition channel, performing speech recognition on the speech segments to obtain speech text; calculating the similarity between the speech text and each first preset keyword in a first preset keyword set, and determining the intent category as a first preset intent if there is a similarity greater than or equal to a first preset similarity threshold; calculating the similarity between the speech text and each second preset keyword in a second preset keyword set, and determining the intent category as a second preset intent if there is a similarity greater than or equal to a first preset similarity threshold; and determining the intent category as a third preset intent if there is no similarity greater than or equal to a first preset similarity threshold in either the first preset keyword set or the second preset keyword set.

[0010] Furthermore, after determining the intent category as the third preset intent, the method includes: monitoring the number of times the third preset intent occurs within a preset time period when the intent category is the third preset intent; generating prompt words and monitoring the passenger's response voice when the number of occurrences is greater than or equal to a preset threshold and other voice segments are detected, wherein the prompt words are used to guide the passenger to respond in order to obtain a response voice; re-identifying the response intent category of the response voice, and modifying the response intent category to the first preset intent or the second preset intent when the response intent category is the third preset intent.

[0011] Further, the step of recognizing speech segments and obtaining intent categories based on the target routing channel also includes: when the target routing channel is a speech processing channel, performing speech recognition on the speech segments to obtain speech text; constructing a first function, and constructing a second function and a third function, wherein the first function is used to determine the coherence of the speech segments, the second function is used to determine whether the speech segment and the previous speech segment are from the same speaker, and the third function is used to determine whether the target words in the speech text have referents in the historical discourse of a preset-length historical discourse sliding window, wherein the target words are pronouns and / or indicator words, and the preset length is used to indicate the number of historical discourses in the historical discourse sliding window; processing the speech text based on the first function, the second function, and the third function to determine the target speech text; performing intent recognition on the target speech text to obtain the intent category, wherein the intent recognition includes: a first-level recognition and a second-level recognition, wherein if the confidence of the first-level recognition is less than the confidence threshold, the second-level recognition is entered to determine the intent category.

[0012] Furthermore, the steps for constructing the first function include: calculating the semantic similarity between the speech text and each historical discourse in the historical discourse sliding window; for each semantic similarity, constructing a semantic similarity weight based on a preset attenuation coefficient, a preset length, and the position of the historical discourse corresponding to the semantic similarity in the historical discourse sliding window; and constructing the first function based on all semantic similarities and all semantic similarity weights.

[0013] Furthermore, each speech segment has a corresponding generation timestamp, and each historical discourse has a corresponding historical generation timestamp. The steps for constructing the second and third functions include: determining the target historical discourse based on the generation timestamps and all historical generation timestamps, wherein the target historical discourse is the historical discourse in the historical discourse sliding window that is closest to the generation timestamp of the speech text; extracting the first voiceprint feature vector of the speech segment corresponding to the speech text, and extracting the second voiceprint feature vector of the historical speech segment corresponding to the target historical discourse; constructing a vector similarity calculation function between the first and second voiceprint feature vectors, and constructing the second function based on the vector similarity calculation function and a preset similarity threshold; extracting the target word set from the speech text, and constructing the target word reference set, wherein the target word reference set is used to store the words referred to by the target words found in the historical discourse sliding window; and constructing the third function based on the target word set and the target word reference set.

[0014] According to another aspect of the embodiments of this application, a voice command scheduling device is also provided, comprising: an acquisition unit, configured to acquire the audio stream of a passenger monitored by a self-service baggage check-in device, and acquire the current locking state of the business operation interface of the self-service baggage check-in device; a routing unit, configured to perform voice activity detection on the audio stream to obtain voice segments, and route the voice segments based on a preset routing function to determine a target routing channel, wherein the preset routing function is used to control the voice segments to enter the target routing channel based on the current locking state; and a recognition unit, configured to recognize the voice segments based on the target routing channel to obtain an intent category, and create a voice command task based on the current locking state and the intent category. The system comprises a task queue, where each voice command task has a corresponding task status; a judgment unit is used to determine the failure status of each historical voice command task in the task queue, obtain the judgment result, and replace the historical voice command task with a new voice command task based on the judgment result; the task queue stores a set of tasks to be scheduled selected from the task registry, which includes voice command tasks with the status of "created" and "queued"; the task registry stores multiple historical voice command tasks that have been created; and a scheduling unit is used to determine the scheduling priority of each historical voice command task in the task queue and schedule the voice command task based on the scheduling priority of all historical voice command tasks.

[0015] Furthermore, the voice command scheduling device includes: a first construction module, used to construct a finite state automaton before obtaining the current locking state of the business operation interface of the self-service baggage check-in device, wherein the finite state automaton includes: a set of locked states, a set of events, a state transition function, an initial locked state, and a mapping function. The set of locked states includes: an idle state, a soft locked state, and a hard locked state. The set of events includes: multiple business events triggered by the passenger performing business operations on the self-service baggage check-in device. The state transition function is used to transition the current locked state based on the business events in the event set.

[0016] Furthermore, the voice command scheduling device also includes: a first judgment module, used to determine the duration of the current lock state being in a hard lock state when the current lock state of the self-service baggage check-in device is a hard lock state after obtaining the current lock state of the business operation interface of the self-service baggage check-in device; and a first conversion module, used to convert the hard lock state to a soft lock state and roll back unfinished business events when the duration exceeds the preset allowed duration.

[0017] Further, the recognition unit includes: a first recognition module, used to perform speech recognition on the speech segment to obtain speech text when the target routing channel is an intent recognition channel; a first calculation module, used to calculate the similarity between the speech text and each first preset keyword in a first preset keyword set, and determine the intent category as a first preset intent when there is a similarity greater than or equal to a first preset similarity threshold; a second calculation module, used to calculate the similarity between the speech text and each second preset keyword in a second preset keyword set, and determine the intent category as a second preset intent when there is a similarity greater than or equal to a first preset similarity threshold; and a first determination module, used to determine the intent category as a third preset intent when there is no similarity greater than or equal to a first preset similarity threshold in either the first preset keyword set or the second preset keyword set.

[0018] Furthermore, the voice command scheduling device also includes: a first monitoring module, used to monitor the number of times the third preset intent occurs within a preset time period after the intent category is determined to be the third preset intent; a first generation module, used to generate prompt words and monitor the passenger's response voice when the number of occurrences is greater than or equal to a preset threshold and other voice segments are detected, wherein the prompt words are used to guide the passenger to respond in order to obtain a response voice; and a first modification module, used to re-identify the response intent category of the response voice and modify the response intent category to the first preset intent or the second preset intent when the response intent category is the third preset intent.

[0019] Furthermore, the recognition unit also includes: a second recognition module, used to perform speech recognition on speech segments to obtain speech text when the target routing channel is a speech processing channel; a second construction module, used to construct a first function, a second function, and a third function, wherein the first function is used to determine the coherence of the speech segment, the second function is used to determine whether the speech segment and the previous speech segment are from the same speaker, and the third function is used to determine whether the target word in the speech text has a referent in the historical discourse of a preset-length historical discourse sliding window, wherein the target word is a pronoun and / or an indicator word, and the preset length is used to indicate the number of historical discourses in the historical discourse sliding window; a second determination module, used to process the speech text based on the first function, the second function, and the third function to determine the target speech text; and a third recognition module, used to perform intent recognition on the target speech text to obtain the intent category, wherein the intent recognition includes: a first-level recognition and a second-level recognition, wherein if the confidence level of the first-level recognition is less than the confidence threshold, the second-level recognition is entered to determine the intent category.

[0020] Furthermore, the second construction module includes: a first calculation submodule, used to calculate the semantic similarity between the speech text and each historical discourse in the historical discourse sliding window; a first construction submodule, used to construct a semantic similarity weight for each semantic similarity based on a preset attenuation coefficient, a preset length, and the position of the historical discourse corresponding to the semantic similarity in the historical discourse sliding window; and a second construction submodule, used to construct a first function based on all semantic similarities and all semantic similarity weights.

[0021] Furthermore, each speech segment has a corresponding generation timestamp, and each historical discourse has a corresponding historical generation timestamp. The second construction module also includes: a first determination submodule, used to determine the target historical discourse based on the generation timestamp and all historical generation timestamps, wherein the target historical discourse is the historical discourse in the historical discourse sliding window that is closest to the generation timestamp of the speech text; a first extraction submodule, used to extract the first voiceprint feature vector of the speech segment corresponding to the speech text, and extract the second voiceprint feature vector of the historical speech segment corresponding to the target historical discourse; a third construction submodule, used to construct a vector similarity calculation function between the first voiceprint feature vector and the second voiceprint feature vector, and construct a second function based on the vector similarity calculation function and a preset similarity threshold; a second extraction submodule, used to extract the target word set in the speech text, and construct a target word reference set, wherein the target word reference set is used to store the words referred to by the target words found in the historical discourse sliding window; and a fourth construction submodule, used to construct a third function based on the target word set and the target word reference set.

[0022] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the scheduling method of any of the above-mentioned voice commands.

[0023] According to another aspect of the embodiments of this application, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any of the above-described voice command scheduling methods.

[0024] In this invention, the audio stream of passengers monitored by the self-service baggage check-in equipment is acquired, and the current locking status of the business operation interface of the self-service baggage check-in equipment is obtained; voice activity detection is performed on the audio stream to obtain voice segments, and the voice segments are routed based on a preset routing function to determine the target routing channel, wherein the preset routing function is used to control the voice segments to enter the target routing channel based on the current locking status; based on the target routing channel, the voice segments are identified to obtain the intent category, and a voice command task is created based on the current locking status and the intent category, wherein the voice command task corresponds to a task status; each historical voice command task in the task queue to be scheduled is invalidated, and the determination result is obtained. Based on the judgment results, historical voice command tasks are replaced with new voice command tasks. The task queue to be scheduled stores a set of tasks to be scheduled selected from the task registry. The task queue includes voice command tasks in the created and queued states. The task registry stores multiple historical voice command tasks that have been created. The scheduling priority of each historical voice command task in the task queue is determined. Based on the scheduling priority of all historical voice command tasks, the voice command tasks are scheduled. This solves the technical problem in related technologies where concurrent control and coordination between voice commands and business operations cannot be achieved, resulting in the inability to accurately schedule voice commands for voice task processing according to the current business operation status.

[0025] In this invention, by acquiring the passenger audio stream and the current locking status of the equipment's business operation interface in real time, the voice segments are intelligently distributed to the corresponding target routing channels using a preset routing function, thereby realizing the linkage between business status and voice processing. Then, based on the target routing channel, the voice segments are identified to obtain the intent category. Based on the identified intent category and the current locking status, voice command tasks are created, and the scheduling tasks can be sorted and executed according to the scheduling priority calculated by multi-factor weighting. This solves the problems of command conflict, accidental interruption of key operations, and chaotic concurrent processing caused by the lack of business status coordination in the existing self-service terminal voice interaction, and realizes accurate coordination and safe scheduling of voice commands and business operations. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0027] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a scheduling method for voice commands is shown.

[0028] Figure 2 This is a flowchart of a voice command scheduling method according to Embodiment 1 of this application;

[0029] Figure 3 This is a flowchart illustrating the lifecycle state transition of a voice command task according to an embodiment of this application;

[0030] Figure 4 This is a schematic diagram of the state transition of a finite state automaton according to an embodiment of this application;

[0031] Figure 5 This is a flowchart of a pre-routing determination according to an embodiment of this application;

[0032] Figure 6 This is a flowchart of a discourse filtering method according to an embodiment of this application;

[0033] Figure 7 This is a flowchart of a multi-factor priority calculation according to an embodiment of this application;

[0034] Figure 8 This is a schematic diagram of a dual-channel event distribution process based on a vector clock according to an embodiment of this application;

[0035] Figure 9 This is a schematic diagram of a voice command interaction system according to an embodiment of this application;

[0036] Figure 10 This is a schematic diagram of a voice command scheduling device according to an embodiment of this application;

[0037] Figure 11 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] It should be noted that all relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected and involved in this invention are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data comply with the relevant laws, regulations, and standards of the relevant regions, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse. For example, this system has an interface with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface. After receiving consent from the aforementioned user or organization, the relevant information is obtained. If the user chooses to refuse, the process enters the expert decision-making process.

[0041] The present invention will now be described in detail with reference to various embodiments.

[0042] Example 1

[0043] According to an embodiment of this application, an embodiment of a voice command scheduling method is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0044] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a voice command scheduling method is shown. Figure 1 As shown, computer terminal 10 (or mobile device) may include one or more ( Figure 1 The processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions may also be included. In addition, it may include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera, wherein the network interface can be connected to wired and / or wireless networks. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0045] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0046] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the voice command scheduling method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the aforementioned voice command scheduling method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0047] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0048] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0049] Under the aforementioned operating environment, this application provides the following: Figure 2 The method for scheduling voice commands is shown. Figure 2 This is a flowchart of the voice command scheduling method according to Embodiment 1 of this application, as follows: Figure 2 As shown, the method includes the following steps:

[0050] Step S201: Obtain the audio stream of passengers monitored by the self-service baggage check-in device, and obtain the current lock status of the business operation interface of the self-service baggage check-in device.

[0051] In this embodiment of the invention, the audio stream is the continuous sound collected in real time by the microphone array of the self-service baggage check-in device, and the current lock status is the current status of the business operation interface maintained by the business lock manager, including idle status, soft lock status and hard lock status, reflecting the security level and process stage of the current business operation.

[0052] Step S202: Perform voice activity detection on the audio stream to obtain voice segments, and route the voice segments based on a preset routing function to determine the target routing channel. The preset routing function is used to control the voice segments to enter the target routing channel based on the current locking state.

[0053] In this embodiment of the invention, speech activity detection is a technique for detecting the start and end boundaries of human voice segments in an audio stream. It is used to remove silent segments and noise, extracting speech segments containing valid human voices. The preset routing function is a pre-constructed segmentation function. Its input is a speech segment and the current locking state, and its output is the target routing channel. This function routes the speech segments to determine the target routing channel. The routing channel may include an automatic speech recognition channel (i.e., a speech processing channel), a confirmation intent rapid recognition channel (i.e., an intent recognition channel), and the target routing channel is one of them.

[0054] Preset routing function It can be defined as:

[0055]

[0056] Where v represents the detected speech segment, L represents the current lock state, FORWARD_ASR represents the automatic speech recognition channel, FORWARD_CIR represents the confirmation intent fast recognition channel, IDLE represents the idle state, SOFT_LOCK represents the soft lock state, and HARD_LOCK represents the hard lock state.

[0057] Step S203: Based on the target routing channel, the voice segment is recognized to obtain the intent category, and a voice command task is created based on the current locking state and the intent category. The voice command task has a corresponding task state.

[0058] In this embodiment of the invention, the intent category is the classification result after semantic understanding of the speech content, such as CONFIRM, CANCEL, QUERY, OPERATION, EMERGENCY, etc. The task status is the state of the voice command task in its lifecycle, including Created, Queued, Executing, Completed, Invalidated, Cancelled, etc.

[0059] In this embodiment of the invention, if the target routing channel is a voice processing channel (i.e., FORWARD_ASR), for voice segments routed to the voice processing channel, an additional tagging function M(v,L)=NORMAL, if L=IDLE; M(v,L)=DEFERRED, if L=SOFT_LOCK.

[0060] It should be noted that the tagging function M(v,L) only applies to speech segments routed to the speech processing channel (i.e., when L=IDLE or L=SOFT_LOCK). When L=HARD_LOCK, the speech segment is routed to the confirmation intent fast recognition channel instead of entering the speech processing channel, so the M(v,L) tag does not apply. This tag can be passed along with the speech segment to the speech command task creation stage. If M(v,L)=NORMAL, the generated task enters Created and can be understood and called after creation; if M(v,L)=DEFERRED, the generated task directly enters Queued.

[0061] Voice command tasks can be created based on the current lock status and intent category. Voice command tasks can include attributes such as: task identifier, original voice text, intent category, priority, task status, creation timestamp, lock level at creation (i.e., lock status), task identifier, and survival time threshold. Among them, priority includes low priority, normal, high priority, and urgent.

[0062] Step S204: Perform an invalidation determination on each historical voice command task in the task queue to be scheduled, obtain the determination result, and replace the historical voice command task with a voice command task based on the determination result. The task queue to be scheduled stores a set of tasks to be scheduled selected from the task registry. The set of tasks to be scheduled includes voice command tasks with the task status of "created" and "queued". The task registry stores multiple historical voice command tasks that have been created.

[0063] In this embodiment of the invention, historical voice command tasks are old tasks that already exist in the task queue but have not yet been executed or have expired. The task registry stores all historical voice command tasks that have been created. The task queue is a subset of the task registry and is a set of historical voice command tasks that are in the created or queued state.

[0064] Failure determination can include intent to replace similar tasks, timeouts, and lock level compatibility checks. Replacement involves removing older, invalid tasks from the scheduling queue or marking them as invalid, thereby freeing up resources or logical space for new tasks.

[0065] When a new voice command task T_new is created, all tasks in the queue of tasks to be scheduled are invalidated, including three invalidation conditions: (1) Intent-like replacement condition: Determine the parent-child / ancestor relationship of the new task (i.e., what task the new task depends on for execution). When the old and new voice command tasks belong to the same intent category, and the old voice command task is not the parent-child / ancestor of the new voice command task, the old voice command task is replaced by the new voice command task and becomes invalid; (2) Timeout condition: When the survival time of the old voice command task exceeds its preset survival time threshold, it automatically becomes invalid; (3) Locking state compatibility condition: When the current locking state of the old voice command task is inconsistent with the locking state at the time of creation, and the current locking state does not allow the old voice command task to execute, the old voice command task becomes invalid. Whether the current locking state allows the old voice command task to execute can be determined by the admission determination function Admit(L,ι). The admission determination function can be:

[0066]

[0067] Where L represents the current lock state and ι represents the intent category. When the current lock state is idle or soft lock, the admission decision function determines that the old voice command task can be executed (i.e., true). When the current lock state is hard lock and the intent category is CONFIRM, CANCEL, or EMERGENCY, the admission decision function determines that the old voice command task can be executed. Only when the current lock state is hard lock and the intent category is not CONFIRM, CANCEL, or EMERGENCY, the admission decision function determines that the old voice command task cannot be executed (i.e., false).

[0068] If any failure condition is met, the lifecycle state of the old voice command task will be changed to Invalidated, which means that the old voice command task is in an unrecoverable failure final state.

[0069] When a new voice command task arrives, the system checks existing historical tasks in the queue and determines whether the new task should overwrite, interrupt, or give way to the historical task through failure detection. This resolves the concurrency problem of voice commands and ensures the continuity and security of business processes.

[0070] To distinguish between persistent lock state changes and transient protective lock state upgrades, a lifecycle state Suspended can be added: When a business process enters an irreversible or high-risk operation node and triggers a lock state upgrade event e_escalate, and the expected duration of HARD_LOCK ΔT is greater than τ_persist, this lock state change is considered a persistent change, and the lifecycle state ι_old of the old voice command task can be modified to Invalidated. Here, τ_persist is taken as τ_max / 2 by default, and τ_max is the maximum allowed duration, with a default value of 60s. When a business process enters an irreversible or high-risk operation node and triggers a lock level upgrade event e_escalate, and the expected duration of HARD_LOCK ΔT is less than or equal to τ_persist (i.e., transient protective switching), the lifecycle state ι_old of the old voice command task can be modified to Suspended instead of Invalidated. Suspended tasks remain in the queue without releasing scheduling resources or sending interrupts. When the lock state falls back to L' (such as SOFT_LOCK or IDLE), and Admit(L', ι_old) = true, and the old voice command task has not triggered a timeout condition, the lifecycle state of the old voice command task is automatically restored to Queued. This mechanism avoids excessive failure of harmless commands that are queued under SOFT_LOCK and mistakenly killed by temporary upgrades under HARD_LOCK (such as passenger-inserted QUERY commands like "Where is the restroom?").

[0071] Figure 3 This is a flowchart illustrating the lifecycle state transition of a voice command task according to an embodiment of this application, such as... Figure 3As shown, the lifecycle states of a voice command task include: Created, Queued, Executing, Completed, Invalidated, Cancelled, and Suspended. When the current locked state L=IDLE, voice command tasks in the created state can be converted to the executing state. When the current locked state L=SOFT_LOCK, voice command tasks in the created state can be converted to the queued state. At the same time, tasks in the queued state can also be restored to the executing state. When a persistent change occurs (i.e., the business process enters an irreversible or high-risk operation node and triggers the lock state upgrade event e_escalate, and the expected duration of HARD_LOCK ΔT is greater than τ_persist), it can be converted to the invalid state. When a transient protective switch occurs (i.e., when the business process enters an irreversible or high-risk operation node and triggers the lock level upgrade event e_escalate, and the expected duration of HARD_LOCK ΔT is less than or equal to τ_persist), it is converted to the suspended state. When the locked state falls back to L' (such as SOFT_LOCK or IDLE), and Admit(L', ι_old)=true, and the old voice command task has not triggered the timeout condition, the lifecycle state of the old voice command task is automatically restored to the queued state. After the suspended task times out, the task state is converted to the invalid state. For a task in the execution state, if the task has been completed, its lifecycle state can be changed to the completed state; if the task has been cancelled, its lifecycle state can be changed to the cancelled state.

[0072] In addition, to avoid the resource stagnation problem of queued tasks when no new voice commands enter for a long time (i.e., the fallback defect of lazy evaluation), the background running daemon thread periodically scans the queue of tasks to be scheduled, and actively invalidates and interrupts speech synthesis for tasks that meet the timeout conditions, ensuring that even in extreme scenarios where no new commands trigger the timeout condition judgment, expired tasks can be cleaned up in a timely manner.

[0073] Step S205: Determine the scheduling priority of each historical voice command task in the task queue to be scheduled, and schedule the voice command task based on the scheduling priority of all historical voice command tasks.

[0074] In this embodiment of the invention, voice command tasks can be scheduled based on the scheduling priority of all historical voice command tasks. Furthermore, based on intent recognition, a multi-factor weighted dynamic priority calculation function can be constructed. The weights satisfy the normalization constraint. ( =0.35, =0.40, =0.25), This is an additional item for emergency intent (unweighted factor). Based on this function, the scheduling priority of each historical voice command task in the task queue to be scheduled can be determined. The sub-item functions can be defined as follows:

[0075] Intent function : W_intent(EMERGENCY)=1, W_intent(CONFIRM)=0.8, W_intent(CANCEL)=0.8, W_intent(OPERATION)=0.5, W_intent(QUERY)=0.3.

[0076] Locked State - Intent Interaction Function :W_lock(HARD_LOCK,ι)=1.0,if ι∈{CONFIRM,CANCEL}; W_lock(HARD_LOCK,ι)=0.0,if ι {CONFIRM,CANCEL,EMERGENCY}; W_lock(SOFT_LOCK,ι)=0.6·Rel(ι,C_current); W_lock(IDLE,ι)=0.5. Where, Rel(ι,C_current) is a semantic similarity function used to calculate the semantic similarity between the current instruction intent and the semantic description (i.e., C_current) of the business being executed by the self-service baggage check-in device:

[0077]

[0078] Here, Emb is a semantic embedding function that maps the semantic description of intent and business into embedding vectors in vector space.

[0079] The context-sensitive function is: ,in, The timestamp of the passenger's most recent interaction with the self-service baggage check-in machine. This is the current timestamp. The time decay coefficient (e.g., λ=0.05) is used to cause the instruction priority to decay naturally after a long period of no interaction.

[0080] Emergency Intent Additional Function Independent of the weighted system, it is directly superimposed as an additional term: U(EMERGENCY)=1, U(others)=0.

[0081] Priority P(T) to discrete priority level The mapping is:

[0082]

[0083] Optionally, when an operation class intent with low semantic relevance (i.e., Rel < θ_rel, default θ_rel = 0.4) is detected in the SOFT_LOCK state, an intent switching confirmation is triggered, which can be triggered under the following conditions:

[0084]

[0085] in, It is a newly identified intent. It is a semantic description of the business process currently being executed. This is the association threshold, and L is the current locking state. This means that both conditions must be met simultaneously. Only when the current lock state is SOFT_LOCK and the correlation between the new intent and the current service is lower than the correlation threshold will the passenger be asked whether to switch services, which triggers the intent switch confirmation.

[0086] In summary, by constructing a multi-level business locking mechanism based on deterministic finite state automata, the business state is divided into idle, soft lock, and hard lock. The audio stream and current locking state of the self-service baggage check-in device are obtained. Through a preset routing function, voice segments are intelligently routed to the automatic voice recognition channel or the confirmation intent rapid recognition channel according to the locking state, realizing pre-filtering for business state perception. Based on the routing results, the intent category is identified and voice command tasks with task status are created. Through multi-dimensional failure judgment, the historical tasks in the task queue to be scheduled are checked for intent replacement, timeout, and lock level compatibility, so that the new task can replace or suspend conflicting old tasks. Finally, based on multi-factor weighted calculation of scheduling priority, historical tasks and new tasks are dynamically sorted and scheduled. This solves the technical problem in related technologies that cannot realize concurrent control and coordination between voice commands and business operations, resulting in the inability to accurately schedule voice commands for voice task processing according to the current business operation status.

[0087] To accurately construct a finite state automaton, the voice command scheduling method provided in Embodiment 1 of this application constructs a finite state automaton, wherein the finite state automaton includes: a locked state set, an event set, a state transition function, an initial locked state, and a mapping function. The locked state set includes: an idle state, a soft locked state, and a hard locked state. The event set includes: multiple business events triggered by passengers performing business operations on self-service baggage check-in equipment. The state transition function is used to transition the current locked state based on the business events in the event set.

[0088] In embodiments of the present invention, such as Figure 4 As shown, Figure 4This is a schematic diagram of the state transition of a finite state automaton according to an embodiment of this application. The business operation interface lock can be modeled as a finite state automaton BLA=(S,E,δ,s0,G), where S is the set of locked states, including IDLE (idle state), SOFT_LOCK (soft lock state), and HARD_LOCK (hard lock state), E is the set of events, including start soft lock event (e_start_soft), start hard lock event (e_start_hard), business completion event (e_complete), cancellation event (e_cancel), timeout event (e_timeout), escalate event (e_escalate), etc., δ is the state transition function, s0 is the initial state, generally IDLE, and G is the state-policy mapping function.

[0089] exist Figure 4 In this context, the locked state can be transitioned using state transition functions. The specific transition rules for the state transition functions are as follows: δ(IDLE, e_start_soft) = SOFT_LOCK; δ(IDLE, e_start_hard) = HARD_LOCK; δ(SOFT_LOCK, e_escalate) = HARD_LOCK; δ(SOFT_LOCK, e_complete) = IDLE; δ(SOFT_LOCK, e_cancel) = IDLE; δ(HARD_LOCK, e_complete) = {SOFT_LOCK, if Q ≠ ;IDLE,if Q= };δ(HARD_LOCK,e_timeout)=SOFT_LOCK. Where Q represents the current queue of tasks to be scheduled, including tasks in the Created and Queued states, tasks in the Suspended state due to the transient HARD_LOCK, and tasks that meet the conditions in the admission decision function after the lock state reverts (i.e., tasks that can resume execution). Completed, canceled, expired, or timed-out tasks are not included in Q. When Q is not empty, it indicates that there are still tasks to be processed after the hard-locked task has completed, and the status is downgraded from HARD_LOCK to SOFT_LOCK; when Q is empty, it indicates that there are no tasks to be processed, and the status is downgraded from HARD_LOCK to IDLE.

[0090] The state-policy mapping function maps each lock state to a corresponding voice command admission policy, which includes policies such as routing voice segments based on the current lock state to determine the target routing channel.

[0091] To prevent deadlock in a hard-locked state, in the voice command scheduling method provided in Embodiment 1 of this application, when the current lock state of the self-service baggage check-in device is a hard-locked state, the duration of the current lock state in the hard-locked state is determined; if the duration exceeds the preset allowed duration, the hard-locked state is converted to a soft-locked state, and unfinished business events are rolled back.

[0092] In this embodiment of the invention, to prevent deadlock in a hard-locked state, a timeout protection mechanism is set up. For a hard-locked state, a timeout degradation function can be constructed: T_guard={s→SOFT_LOCK,if s=HARD_LOCK∧(t_now-t_enter)>τ_max}, where t_now is the current timestamp, t_enter is the timestamp of entering the hard-locked state, and τ_max is the maximum allowed duration (default τ_max=60s). That is, when the current lock state of the self-service baggage check-in device is a hard-locked state, the duration of the current lock state (i.e., t_now-t_enter) is determined. If the duration is greater than the preset allowed duration (i.e., τ_max), the hard-locked state is converted to a soft-locked state, and unfinished business events are rolled back.

[0093] To avoid hard lock timeout degradation and state fragmentation in the underlying business system, when the e_timeout event is triggered, in addition to driving the state transition function δ(HARD_LOCK,e_timeout)=SOFT_LOCK, a cancellation event must also be broadcast to the backend business service layer simultaneously to force rollback or terminate the unfinished transactions of the current business b (e.g., canceling payment suspension, releasing baggage check-in space, reclaiming identity verification sessions, etc.); or, the backend business service layer must have its own local timeout rollback mechanism that is strictly aligned with τ_max. This ensures that the state of the front-end (i.e., self-service baggage check-in device) interface is strictly consistent with the state view of the underlying business engine at the current timestamp, avoiding the risk of state fragmentation between the front-end and back-end caused by the front-end lock state being downgraded while the backend business is still suspended.

[0094] For example, a passenger makes a payment at a self-service baggage check-in machine, and the machine enters a hard-lock state. Due to a network failure, the payment service does not return a result within the specified time, and the passenger does not make any voice input. At this point, the hard-lock state lasts for 70 seconds, exceeding the preset allowed time of 60 seconds. Upon detecting the timeout, the state is immediately changed from hard-lock to soft-lock, and a cancellation request is sent to the payment service to terminate the payment process and release related resources. Subsequently, an announcement is made: "Payment timed out, please try again," guiding the passenger into a new interaction process, thereby preventing the device from getting stuck on the payment confirmation screen.

[0095] To accurately determine the intent category, in the voice command scheduling method provided in Embodiment 1 of this application, when the target routing channel is an intent recognition channel, voice segment is subjected to voice recognition to obtain voice text; the similarity between the voice text and each first preset keyword in the first preset keyword set is calculated, and if there is a similarity greater than or equal to the first preset similarity threshold, the intent category is determined as the first preset intent; the similarity between the voice text and each second preset keyword in the second preset keyword set is calculated, and if there is a similarity greater than or equal to the first preset similarity threshold, the intent category is determined as the second preset intent; if there is no similarity greater than or equal to the first preset similarity threshold in either the first preset keyword set or the second preset keyword set, the intent category is determined as the third preset intent.

[0096] In this embodiment of the invention, for voice segments routed to the intent recognition channel (i.e., FORWARD_CIR) (which is only routed to during hard-locked states), FORWARD_CIR executes a fast intent determination function. , This function can be implemented based on a lightweight keyword matching model, which maintains a confirmation keyword set K_confirm (i.e., the first preset keyword set) and a cancellation keyword set K_cancel (i.e., the second preset keyword set). The specific intent determination rules are as follows:

[0097]

[0098] Where k represents the keywords matched based on the lightweight keyword matching model, and v represents the speech segment. The text is based on a lightweight speech recognition model, and sim is the string edit distance similarity function (normalized Levenshtein distance). To confirm the intent recognition threshold (default) When Ψ(v)=REJECT, the speech segment is discarded and a digital human prompt is triggered.

[0099] In this embodiment of the invention, the first preset intention is... The second presupposed intention is The third presupposed intention is .

[0100] To improve the accuracy of digital human interaction, in the voice command scheduling method provided in Embodiment 1 of this application, when the intent category is a third preset intent, the number of times the third preset intent appears within a preset time period is monitored; when the number of occurrences is greater than or equal to a preset threshold and other voice segments are detected, prompt words are generated and the passenger's response voice is monitored, wherein the prompt words are used to guide the passenger to respond in order to obtain the response voice; the response intent category of the response voice is re-identified, and when the response intent category is a third preset intent, the response intent category is modified to a first preset intent, or the response intent category is modified to a second preset intent.

[0101] Because passengers often use colloquial expressions outside the vocabulary list (such as "okay," "do it," "no problem," "that's it") in key confirmation scenarios, simply relying on hard matching with a fixed vocabulary list can easily lead to continuous REJECT intents, resulting in an interactive deadlock of repeated prompts. To address this, a degradation retry chain is constructed: a continuous REJECT count n_reject is maintained for the same key confirmation node. When n_reject ≥ N_thr (i.e., a preset threshold number of times, with N_thr=2 by default), that is, when the number of times the third preset intent appears within a preset time period (e.g., within a continuous time period of the same key confirmation node) is monitored, and when continuous voice input (i.e., other voice segments) is detected from the passenger, a two-level degradation strategy is triggered. In the first level, the digital human on the self-service baggage check-in device prioritizes explicit prompts to regulate passenger vocabulary (e.g., please clearly answer confirm or cancel), guiding colloquial expressions within the vocabulary coverage. In the second level, if the response intent category still shows REJECT (i.e., the third preset intent) after the first-level prompt, the single voice segment can optionally and temporarily be allowed to enter the deep semantic intent determination step. This step only determines the CONFIRM and CANCEL tags (excluding OPERATION, QUERY, etc. intents), that is, modifying the response intent category to the first preset intent or the second preset intent. After the determination is completed, the channel is immediately closed and control is returned to the intent recognition channel. This degradation retry chain breaks the interaction deadlock without violating the overall HARD_LOCK security semantics (i.e., OPERATION and QUERY intents are always pre-filtered), reducing the probability of interaction failure caused by vocabulary rigidity.

[0102] Optionally, n_reject is cleared when confirming intent (CONFIRM), canceling intent (CANCEL), or when the session ends.

[0103] Figure 5 This is a flowchart of a pre-route determination according to an embodiment of this application, such as... Figure 5As shown, the detected voice segments and the obtained current lock level (i.e., current lock state) are sent to the routing function for judgment. If the current lock state is a hard lock (i.e., L=HARD_LOCK), the decision is to route to the confirmation intent fast recognition channel (i.e., FORWARD_CIR) for fast intent recognition. This channel maintains a multilingual keyword library: Chinese confirmation word library K_confirm_zh={confide, confirm, okay, agree, yes, right, no problem,...}, English confirmation word library K_confirm_en={confirm, yes, ok, sure, agree, right,...}; Chinese cancellation word library K_cancel_zh={cancel, don't, return, forget it, no,...}, English cancellation word library K_cancel_en={cancel, no, back, stop, abort,...}. This channel can recognize three types of intents: CONFIRM / CANCEL / REJECT. If the intent quickly identified is REJECT, a continuous REJECT counter n_reject can be maintained for the same key confirmation node. The system checks if the number of REJECTs (n_reject) is greater than or equal to a threshold (N_thr, e.g., 2) and if there is still continuous voice input. If the number of REJECTs is less than the threshold, the digital human can be triggered to provide feedback and re-identify the intent. If n_reject ≥ N_thr and continuous voice input from the passenger is detected (i.e., REJECTs are identified twice consecutively (including after feedback), a downgrade is triggered. In the first level, the digital human issues an explicit vocabulary suggestion (e.g., "Please answer clearly to confirm or cancel"), guiding the conversational expression to the vocabulary coverage area, and waiting for the passenger to respond again. In the second level, if a REJECT still appears after the first-level suggestion, the single voice segment is temporarily allowed to enter the deep semantic judgment channel. In this downgraded channel, only CONFIRM / CANCEL tags are judged (OPERATION / QUERY intent recognition is not enabled). After the judgment is completed, the channel is immediately closed, and control is returned to the confirmation intent quick recognition channel. If the current locking state is soft-locked (i.e., L=SOFT_LOCK), the routing is determined to the voice processing channel (i.e., FORWARD_ASR). If M(v,L)=DEFERRED for this channel, the created task directly enters the Queued state and is placed in the queue. If the current locking state is soft-locked (i.e., L=IDIE), the routing is determined to the voice processing channel (i.e., FORWARD_ASR). If M(v,L)=NORMAL for this channel, the created task enters the Created state and is scheduled according to the normal rules (i.e., it can be executed immediately).

[0104] To improve the accuracy of determining the intent category, in the voice command scheduling method provided in Embodiment 1 of this application, when the target routing channel is a voice processing channel, voice recognition is performed on the voice segment to obtain voice text; a first function is constructed, and a second function and a third function are also constructed. The first function is used to determine the coherence of the voice segment, the second function is used to determine whether the voice segment and the previous voice segment are from the same speaker, and the third function is used to determine whether the target word in the voice text has a referent in the historical discourse of a preset-length historical discourse sliding window. The target word is a pronoun and / or an indicator word, and the preset length is used to indicate the number of historical discourses in the historical discourse sliding window; based on the first function, the second function, and the third function, the voice text is processed to determine the target voice text; intent recognition is performed on the target voice text to obtain the intent category. The intent recognition includes: a first-level recognition and a second-level recognition. If the confidence level of the first-level recognition is less than the confidence threshold, the second-level recognition is entered to determine the intent category.

[0105] In this embodiment of the invention, when the target routing channel is a voice processing channel, voice recognition is performed on the voice segments to obtain voice text. Since the actual operating scenario of self-service baggage check-in equipment is characterized by multiple users sharing the space, noisy backgrounds, and frequent intervention by accompanying persons, for example, when passengers are checking in their baggage, there may be conversations between family members, children playing, and other passengers talking at nearby terminals; passengers' continuous instructions often involve referential relationships (e.g., "Change that one," "I want this one") and sudden topic changes (e.g., "Never mind, I'll go ask the check-in counter first"). Relying solely on business semantic similarity calculation and intent switching confirmation mechanisms is insufficient to fully identify the above scenarios: the former only measures the association between the new intent and the current business, without modeling the continuity relationship between discourses; the latter is only triggered under SOFT_LOCK and does not distinguish between topic switching and speaker switching. Therefore, after performing voice recognition on the voice segments, a sliding historical discourse window is constructed. Based on this historical discourse window, each newly identified voice text is comprehensively evaluated from three dimensions: topic coherence, speaker continuity, and referentiality, and differentiated subsequent processing strategies are triggered based on the evaluation results.

[0106] In this embodiment of the invention, a first function, a second function, and a third function are constructed. The first function is used to determine the coherence of a speech segment (i.e., topic coherence), the second function is used to determine whether the speech segment and the previous speech segment are from the same speaker (i.e., speaker continuity), and the third function is used to determine whether the target word in the speech text has a referent in the historical discourse of a preset-length historical discourse sliding window (i.e., referentiality). The target word is a pronoun and / or an indicator word. The preset length (e.g., k) is used to indicate the number of historical discourses in the historical discourse sliding window.

[0107] Historical discourse sliding window Let u_{n-K+i} represent the i-th historical utterance in the historical utterance sliding window, where i = 1, 2, ..., k. When the current window length is less than K, the new utterance u_{n+1} is directly appended to the end; when the current window is full, the earliest u_{n-K+1} is deleted, the remaining elements are shifted to the left, and then u_{n+1} is placed at the end. The window can be cleared when the session ends, the business context is reset, or the configured long-term idle threshold is exceeded.

[0108] A comprehensive decision function can be constructed, which combines the outputs of the first, second, and third functions to determine the target speech text. The comprehensive decision function is as follows:

[0109]

[0110] in, This is the output of the second function. This is the output of the first function. The output of the third function, This is the threshold for topic coherence, with a default value of 0.55. This indicates a change in the speaker. This indicates that the same passenger has switched topics. The referential relationships in the utterance cannot be resolved. This indicates that the speech is coherent and allows for continued processing.

[0111] It should be noted that only when the output of the comprehensive decision function is COHERENT is the speech text identified as the target speech text and directly proceed to subsequent intent recognition. When the output of the comprehensive decision function is TOPIC_SHIFT, the text is sent to a candidate recognition process that only recognizes intent and does not create a task. After obtaining the new intent, the passenger is asked whether to switch services. Only after confirmation is a voice command task created. When the output of the comprehensive decision function is SPEAKER_DRIFT, voiceprint, questioning, or visual verification is performed first. After confirming that the passenger is not the primary passenger, the speech text is transferred to an isolation queue and does not enter the intent recognition process of the current primary service. When the output of the comprehensive decision function is REF_BROKEN, the digital human initiates a disambiguation question. After the passenger clarifies the information, the complete speech is re-sent to the comprehensive decision function for judgment.

[0112] For example, if a passenger is checking in their luggage but suddenly says, "I'd like to change to a window seat," the DCF will identify this as TOPIC_SHIFT. It can first perform candidate intent recognition to obtain the seat selection intention, but instead of creating a formal task, it will ask, "Do you want to switch from luggage check-in to seat selection?" Only after the passenger confirms will the intent be converted into a formal task. If a companion says, "What should we eat for lunch?" the DCF will determine this as SPEAKER_DRIFT based on the speaker's characteristics and the topic content. After verification, the utterance will be isolated to prevent it from interfering with the primary passenger's current activity. If the passenger only says, "Change that," but the object referred to by "that" cannot be found in the historical utterances, the DCF will determine this as REF_BROKEN, and the digital human will ask, "Which item do you want to change?" After the passenger clarifies, the process will re-enter the DCF and the hierarchical intent recognition process.

[0113] Optionally, to reduce false positives in SPEAKER_DRIFT caused by acoustic disturbances such as coughing, colds, deliberately lowering one's voice, and sudden background noise, a pre-verification step can be added before routing to the isolation queue: when at a critical interaction node (such as L=HARD_LOCK, or when there is an active URGENT priority task in the task queue to be scheduled), the digital human first triggers a lightweight question (such as "Was that you issuing the command just now?") or visual prompt, allowing the passenger to respond immediately with a brief confirmation / cancellation response; if the self-service baggage check-in equipment has camera capabilities, the voice segment can be further cross-verified with the facial visual features at the same time in a multimodal manner. Only when both the voiceprint and visual features are determined to be those of a non-primary passenger will the voice text be transferred to the isolation queue, avoiding the silent swallowing of the primary passenger's command due to a single voiceprint misjudgment, and improving the robustness of DCF to false positives at critical nodes.

[0114] Figure 6 This is a flowchart of a discourse filtering method according to an embodiment of this application, such as... Figure 6As shown, to address anomalies such as casual conversations, linguistic references, and sudden topic shifts in civil aviation self-service scenarios, speech can be filtered. Specifically, a sliding historical speech window H_K of length K (default K=5) is maintained. Each historical speech in the sliding historical speech window can be denoted as u_i=(r_i,ι_i,t_i,spk_i), which are the embedding vectors of the identified text, intent category, timestamp, and speaker's speech, respectively. For each newly identified speech segment and speech text, a comprehensive judgment is made from three dimensions: topic coherence, speaker continuity, and resolvability of reference (i.e., the aforementioned referentiality). When the comprehensive judgment (i.e., the comprehensive judgment function) outputs TOPIC_SHIFT, the text is sent to a candidate recognition process that only recognizes intent and does not create a task. After obtaining the new intent, the passenger is asked whether to switch services. Only after confirmation is a voice command task created. When the comprehensive judgment outputs COHERENT, the voice text is identified as the target voice text, and subsequent intent recognition (i.e., normal processing) is performed directly. When the comprehensive judgment function outputs SPEAKER_DRIFT, a pre-verification is added. At key interaction nodes, a lightweight digital human is first asked to confirm whether the passenger is the primary passenger. If the passenger is confirmed to be the primary passenger, the output is converted to COHERENT. If the digital human does not respond or confirms that it is not the main passenger, the voice text is transferred to the isolation queue (i.e., the non-passenger voice queue). If the self-service terminal has camera capability, the newly recognized voice segment is cross-validated with the facial visual features at the same time using multimodal visual methods. If it is verified as the main passenger, the output is converted to COHERENT. If it is verified as a non-passenger, the voice text is also transferred to the isolation queue. Furthermore, if it is not at a critical interaction node, the voice text is directly transferred to the isolation queue. When the output of the comprehensive judgment function is REF_BROKEN, the digital human first initiates a counter-question. After the passenger clarifies the information, the complete speech is then re-entered into the comprehensive judgment function for judgment.

[0115] Intent recognition for target speech text can be performed using a two-layer cascaded architecture, which includes a first-level recognition and a second-level recognition. If the confidence score of the first-level recognition is less than a confidence threshold (e.g., 0.85), it proceeds to the second-level recognition to determine the intent category. The first-level recognition can be performed using a rule-based fast classifier (e.g., keywords + regular expressions), which only covers high-frequency and simple intents and has low recognition latency. The second-level recognition can be performed using a deep learning-based semantic understanding model to handle complex intents.

[0116] For example, the keyword + regular expression steps can be as follows: First, scan the target speech text to check if it contains any word from the keyword set. If it does, it is initially determined to be a relevant intent. For example, if the target speech text is: "Check the status of flight CA1234", the text contains the keywords "check" and "status", which can be determined to be a query intent. Furthermore, if CA1234 is detected as matching the flight number regular expression, the match is successful. Since both the intent keyword and the explicit entity are matched, a high confidence score of 0.9 is assigned.

[0117] In this embodiment of the invention, since all speech segments are routed to the speech processing channel in the HARD_LOCK state, they do not enter the intent recognition channel regardless of whether the determination result is CONFIRM, CANCEL or REJECT. Therefore, the intent recognition channel call during the hard-lock state is completely cut off. Compared with the computational overhead of sending all speech segments into the intent recognition channel in the prior art, the computational overhead is reduced and the computational efficiency is improved.

[0118] Figure 7 This is a flowchart of a multi-factor priority calculation according to an embodiment of this application, such as... Figure 7 As shown, the microphone input is continuously monitored, and the detected speech segments and recognized speech text are sent to the business state awareness filter. The business state awareness filter performs speech coherence filtering through a comprehensive decision function. The output of the comprehensive decision function can be: COHERENT, TOPIC_SHIFT, SPEAKER_DRIF, and REF_BROKEN. Only when the output of the comprehensive decision function is COHERENT is the recognized speech text determined as the target speech text, and subsequent intent recognition is directly performed: First, a simple intent recognition is performed through a rule-based fast classifier, and it is determined whether the confidence of the simple intent recognition is greater than or equal to a threshold. If the confidence is greater than or equal to the threshold, the intent of the speech text is obtained. If the confidence is less than the threshold, a semantic language understanding model is further used to process complex intents to obtain the intent of the speech text. Then, a multi-factor weighted priority function is constructed to calculate the priority of the speech command task corresponding to the speech text, and priority mapping is performed, that is, the priority is mapped to discrete priority levels (including URGENT, HIGH, NORMAL, and LOW).

[0119] To accurately determine the target speech text, a first function needs to be constructed first. In the speech command scheduling method provided in Embodiment 1 of this application, the semantic similarity between the speech text and each historical discourse in the historical discourse sliding window is calculated. For each semantic similarity, a semantic similarity weight is constructed based on a preset attenuation coefficient, a preset length, and the position of the historical discourse corresponding to the semantic similarity in the historical discourse sliding window. The first function is constructed based on all semantic similarities and all semantic similarity weights.

[0120] In this embodiment of the invention, the first function is:

[0121] ;

[0122] in, It is voice text. These are K historical discourse windows (i.e., historical discourse sliding windows). It is the text of the i-th historical discourse in the window. This indicates that the text is mapped to a semantic embedding vector. This means mapping the speech text to a semantic embedding vector, and cos means calculating the cosine similarity between the two vectors (that is, calculating the semantic similarity between the speech text and each historical discourse in the sliding window of historical discourse). This is the normalized time decay weight (i.e., semantic similarity weight) of the i-th historical discourse. The larger the value of the first function, the more consistent the new discourse is with the recent dialogue topic. When the historical discourse window is not yet full, it should be summed according to the actual length and the time decay weight should be normalized.

[0123] In an embodiment of the present invention, In this context, the preset attenuation coefficient μ is set to 0.3 by default, i represents the position corresponding to each historical discourse in the historical discourse window, i=1 represents the earliest position, i=K represents the most recent position, k is the preset length (i.e. the number of historical discourses in the historical discourse sliding window), and exp is the natural exponent.

[0124] Each speech segment has a corresponding generation timestamp, and each historical utterance has a corresponding historical generation timestamp. To improve the accuracy of determining the target speech text, a second function and a third function need to be constructed. In the speech command scheduling method provided in Embodiment 1 of this application, the target historical utterance is determined based on the generation timestamp and all historical generation timestamps. The target historical utterance is the historical utterance closest to the generation timestamp of the speech text in the historical utterance sliding window. The first voiceprint feature vector of the speech segment corresponding to the speech text is extracted, and the second voiceprint feature vector of the historical speech segment corresponding to the target historical utterance is extracted. A vector similarity calculation function between the first and second voiceprint feature vectors is constructed, and a second function is constructed based on the vector similarity calculation function and a preset similarity threshold. The target word set in the speech text is extracted, and a target word reference set is constructed. The target word reference set is used to store the words referred to by the target words found in the historical utterance sliding window. The third function is constructed based on the target word set and the target word reference set.

[0125] In this embodiment of the invention, the second function is:

[0126] ;

[0127] in, It is a voice clip. It is the most recent historical statement in the window. This represents the embedded vector (i.e., the first voiceprint feature vector) extracted from the speech segment. This indicates the extraction of the embedding vector (i.e., the second voiceprint feature vector) of the most recent historical utterance. This represents a vector similarity calculation function between the first and second voiceprint feature vectors, used to calculate the cosine or Euclidean distance between the two embedded vectors. It is the threshold for the same speaker (i.e., the preset similarity threshold). [] denotes a conditional function. The function returns 1 if the condition within [] is true, and 0 otherwise. In other words, speaker continuity is determined when the distance between two embedding vectors is less than a threshold. The most recent historical utterance (i.e., the target historical utterance) is determined by the timestamp of the speech segment's generation and the historical timestamps of all historical utterances.

[0128] The third function is:

[0129] ;

[0130] in, This represents the set of pronouns / indicators in spoken text. Let p denote the empty set, and p be a set Any word in it, This represents the set of candidate antecedents (target word reference set) of p retrieved within the historical discourse sliding window. This indicates a logical OR. [] represents a conditional function. It returns 1 if the condition inside [] is true, and 0 otherwise. That is, it returns 1 if there are no pronouns / indicators in the speech text, or if each pronoun / indicator can find at least one antecedent, and 0 otherwise.

[0131] The target words are pronouns and / or demonstrative words, such as those, this, just now, it, that, etc.

[0132] In this embodiment of the invention, when the backend business system performs an operation (e.g., baggage scanning completed, payment successful, etc.), the frontend (i.e., self-service baggage check-in device) needs to know this change immediately in order to change the interaction strategy. This can be achieved by sending events from the server (i.e., the backend business system) through dual channels. The dual channels can be an SSE channel and a WebSocket channel. The SSE channel carries the business status change event set E_sse={LockLevelChangeEvent, TaskStatusChangeEvent, BusinessProgressEvent, SystemAlertEvent}, while the WebSocket channel carries the real-time interactive data stream E_ws={AudioStream_up, ASRResult_down, TTSAudio_down, CommandFeedback_down}. The two channels are associated through a unified session identifier.

[0133] Each event in the dual-channel architecture carries a vector clock, which includes a sequence number component from the first preset channel (i.e., the SSE channel) and a sequence number component from the second preset channel (i.e., the WebSocket channel). Events originate from either the SSE or WebSocket channel. The sequence number component of an event in its source channel must be strictly incremented by 1 to ensure sequential delivery within that channel. Furthermore, the sequence number component in the other channel must not exceed the locally known state (i.e., not exceed the sequence number component corresponding to the event in the other channel already processed by the self-service baggage check-in system) to ensure causal dependencies are satisfied. An event is delivered only if events in the same channel are consecutive and the dependent event in the other channel has been processed; otherwise, the event enters a buffer queue to wait for the missing event (i.e., the dependent event) to arrive. This prevents the incorrect delivery of illegal out-of-order events that are simultaneously one step ahead in both channels.

[0134] For example, each event E is transmitted carrying a vector clock (NA, NB), where NA is the sequence number component of channel A (e.g., SSE channel) and NB is the sequence number component of channel B (e.g., WebSocket channel). This vector clock represents the NAth event received by SSE and the NBth event received by WebSocket. The self-service baggage check-in device has processed event E2 (0, 1) and received the current event E1 (1, 1) from channel A. The self-service baggage check-in device compares the vector clocks of the two events. The sequence number component 0 plus 1 of event E2 equals the sequence number component 1 of the corresponding event E1, and the sequence number component 1 of event E2 equals the sequence number component 1 of the corresponding event E1. That is, in channel A, event E2 needs to be delivered earlier than event E1 (i.e., the sequence number component of the SSE channel is incremented by 1). In channel B, event E2 and event E1 are delivered simultaneously (i.e., not exceeding the locally known state).

[0135] If an event waits in the buffer queue for longer than a preset time threshold without receiving its dependent events, a new vector clock is obtained by taking the maximum value of the previous event processed by the self-service baggage check-in device (i.e., locally) and the received current event, sequentially by number. This time threshold defaults to 500ms. For example, if the vector clock of the previously processed event locally is (5, 3) and the vector clock of the received current event is (7, 2), taking the maximum value of the sequential components yields a new vector clock of (7, 3), indicating that locally, the SSE channel has synchronized at least the 7th event and the WebSocket channel at least the 3rd event. At this point, the event corresponding to the new vector clock can be forcibly delivered, and state synchronization repair can be performed. For example, a full state synchronization can be triggered, pulling the current complete lock state and task registry snapshot from the server. Forced delivery does not mean confirming that all dependencies have arrived; rather, it first bridges the gap and then corrects it using the server's snapshot.

[0136] During event delivery and full state synchronization, the local (i.e., client-side, self-service baggage check-in device) cached lock state L may momentarily differ from the server's actual state. If a new voice input from a passenger is detected at this time, making a routing decision directly based on the locally expired lock state L may lead to release / interception errors. To eliminate this synchronization blind spot, after triggering full state synchronization, a short-lived Input_Suspend flag is set (to suspend the input voice segment). While this flag is active, the collected voice segments along with their timestamps t_recv are temporarily stored in a local restricted buffer B_input (the maximum capacity is the voice duration corresponding to a preset time threshold, with a default of 500ms of sampled data). They are not immediately sent to the Route(v,L) function for judgment. Once the Input_Suspend flag is deactivated or expires (i.e., the local vector clocks in both channel dimensions are aligned with the server), the voice segments in B_input are re-determined according to their timestamps based on the latest lock state. This mechanism enables voice segments within the vector clock synchronization dead zone to be processed with delay rather than error, extending the dual-channel causal consistency guarantee from the event distribution layer to the routing decision.

[0137] Figure 8 This is a schematic diagram of a dual-channel event distribution process based on a vector clock according to an embodiment of this application, as shown below. Figure 8As shown, the server distributes events through dual channels, including an SSE channel and a WebSocket channel. The client receives events sent through both channels. Events are delivered only when events in the same channel are consecutive and the dependent events in the other channel have been processed (i.e., channel-aware causal deliverability). Otherwise, the event enters a buffer queue to wait. If an event waits in the buffer queue for more than a preset time threshold without receiving its dependent events (i.e., missing events are not received), a timeout-forced delivery is triggered, along with a full state fetch. After triggering the full state fetch, a brief Input_Suspend flag is set (using...). While the input speech segment is suspended, new speech detected during speech activity is not routed. The speech segment, along with its timestamp t_recv, is temporarily stored in a local restricted buffer B_input (the maximum capacity of which is the speech duration corresponding to a preset time threshold, with a default of 500ms of sampled data). It is not immediately sent to the Route(v,L) function for judgment. After the Input_Suspend flag is removed or expires (i.e., after the client and server are aligned in both channels), the speech segments in B_input are re-determined according to their timestamps based on the latest locking status. This mechanism allows speech segments in the vector clock synchronization dead zone to be processed with delay rather than error, extending the guarantee of dual-channel causal consistency from the event distribution layer to the routing decision.

[0138] Figure 9 This is a schematic diagram of a voice command interaction system according to an embodiment of this application, such as... Figure 9As shown, the system comprises four layers: self-service terminal equipment, voice processing layer, business collaboration layer, and backend service layer (multi-agent). The front-end interaction layer is deployed on the self-service terminal equipment and includes a microphone array acquisition module, a voice activity detection module, a business status awareness filter, a confirmation intent rapid recognition module, and a digital human rendering module. The microphone array acquisition module continuously captures ambient audio and transmits it to the voice activity detection module. Upon detecting voice activity, the voice activity detection module generates voice segments and sends them to the business status awareness filter module. This filter module, based on the current global lock state (IDLE / SOFT_LOCK / HARD_LOCK) provided by the business lock manager, distributes the voice segments to the voice processing channel or the confirmation intent rapid recognition module via a preset routing function. If routed to the confirmation intent rapid recognition module (only in the HARD_LOCK state), lightweight keyword matching is performed to quickly determine whether the intent is confirmed or canceled. The decision result, status prompts, or interactive feedback are then presented to the user in voice and visual form through the digital human rendering module. If routed to the voice processing channel, complete voice recognition and intent understanding are performed (i.e., entering the voice processing layer). The speech processing layer includes streaming speech recognition, a speech coherence filter, and hierarchical intent recognition. Streaming speech recognition converts audio streams into text sequences in real time. The speech coherence filter, located between the streaming speech recognition output and the hierarchical intent recognition, maintains a sliding historical speech window and evaluates the topic coherence, speaker continuity, and referential resolvability of new speech in parallel to intercept abnormal speech such as idle chatter or linguistic references. The hierarchical intent recognition adopts a cascaded architecture. First, a rule-based fast classifier processes high-frequency simple intents. If the confidence level is below a threshold, a deep learning-based semantic understanding model processes complex intents. The decision results, status prompts, or interactive feedback are presented to the user in both voice and visual forms through a digital human rendering module. The system triggers corresponding business status changes or task scheduling based on the recognition results, while using a vector clock mechanism to ensure consistency between the front-end display status and the back-end business status. The business collaboration layer includes a business lock manager (i.e., a finite state automaton), a task lifecycle manager (for voice command task creation and failure determination), a command scheduler (for priority calculation), and a dual-channel event dispatcher (for vector clock synchronization). The backend service layer handles specific business logic through a multi-agent orchestration framework (including check-in agent, baggage check-in agent, seat selection agent, payment agent, verification agent, etc.).

[0139] The voice command scheduling method provided in this application can construct a multi-level service locking mechanism based on finite state automata, dividing service states into idle, soft-locked, and hard-locked states, and driving state transitions based on service events to achieve deep collaboration and formal security assurance between voice interaction and service operations. By introducing discourse coherence filtering, new discourse is evaluated from three dimensions: topic coherence, speaker continuity, and referentiality, achieving interference interception and accurate identification of primary passenger commands in multi-user shared scenarios. Through pre-routing and a fast intent recognition channel, the call to heavy voice processing channels is cut off in the hard-locked state, and only lightweight and fast determination is performed on confirmation / cancellation intents, achieving efficient utilization of computing resources and security protection for critical operations. Through multi-dimensional failure judgment and task lifecycle management, concurrent tasks are replaced, suspended, or invalidated based on intent, time, and lock state compatibility, achieving complete resolution of command conflicts. It also uses multi-factor weighted dynamic calculation of priorities and combines vector clock to ensure causal consistency of dual-channel events, realizing intelligent instruction scheduling and strong consistency of state synchronization in complex interaction scenarios. It achieves concurrent control and collaboration between voice commands and business operations, and can accurately schedule voice commands for voice task processing based on the current state of the operation interface.

[0140] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0141] Example 2

[0142] This application also provides a voice command scheduling device. It should be noted that the voice command scheduling device of this application can be used to execute the voice command scheduling method provided in this application. The voice command scheduling device provided in this application will be described below.

[0143] According to an embodiment of this application, an apparatus for implementing the above-described voice command scheduling method is also provided. Figure 10 This is a schematic diagram of a voice command scheduling device according to an embodiment of this application, such as... Figure 10 As shown, the voice command scheduling device may include: an acquisition unit 100, a routing unit 101, a recognition unit 102, a judgment unit 103, and a scheduling unit 104.

[0144] The acquisition unit 100 is used to acquire the audio stream of passengers monitored by the self-service baggage check-in equipment and to acquire the current lock status of the business operation interface of the self-service baggage check-in equipment.

[0145] The routing unit 101 is used to detect voice activity in the audio stream, obtain voice segments, and route the voice segments based on a preset routing function to determine the target routing channel. The preset routing function is used to control the voice segments to enter the target routing channel based on the current locking state.

[0146] The recognition unit 102 is used to recognize speech segments based on the target routing channel, obtain the intent category, and create a voice command task based on the current locking state and the intent category, wherein the voice command task corresponds to a task state.

[0147] The determination unit 103 is used to determine the failure of each historical voice command task in the task queue to be scheduled, obtain the determination result, and replace the historical voice command task with a voice command task based on the determination result. The task queue to be scheduled stores a set of tasks to be scheduled selected from the task registry. The set of tasks to be scheduled includes voice command tasks with the task status of being created and queued. The task registry stores multiple historical voice command tasks that have been created.

[0148] The scheduling unit 104 is used to determine the scheduling priority of each historical voice command task in the task queue to be scheduled, and to schedule the voice command tasks based on the scheduling priorities of all historical voice command tasks.

[0149] The voice command scheduling device provided in this application embodiment can acquire the audio stream of passengers monitored by the self-service baggage check-in equipment through the acquisition unit 100, and acquire the current locking status of the business operation interface of the self-service baggage check-in equipment. It can perform voice activity detection on the audio stream through the routing unit 101 to obtain voice segments, and route the voice segments based on a preset routing function to determine the target routing channel. It can identify the voice segments based on the target routing channel through the recognition unit 102 to obtain the intent category, and create a voice command task based on the current locking status and intent category. It can determine the failure of each historical voice command task in the task queue to be scheduled through the judgment unit 103, obtain the judgment result, and replace the historical voice command task with a voice command task based on the judgment result. It can determine the scheduling priority of each historical voice command task in the task queue to be scheduled through the scheduling unit 104, and schedule the voice command task based on the scheduling priority of all historical voice command tasks.

[0150] Optionally, the voice command scheduling device includes: a first construction module, used to construct a finite state automaton before obtaining the current locking state of the business operation interface of the self-service baggage check-in device, wherein the finite state automaton includes: a set of locked states, a set of events, a state transition function, an initial locked state, and a mapping function. The set of locked states includes: an idle state, a soft locked state, and a hard locked state. The set of events includes: multiple business events triggered by the passenger performing business operations on the self-service baggage check-in device. The state transition function is used to transition the current locked state based on the business events in the event set.

[0151] Optionally, the voice command scheduling device further includes: a first judgment module, used to determine the duration of the current lock state being in a hard lock state when the current lock state of the self-service baggage check-in device is a hard lock state after obtaining the current lock state of the business operation interface of the self-service baggage check-in device; and a first conversion module, used to convert the hard lock state to a soft lock state and roll back unfinished business events when the duration exceeds a preset allowed duration.

[0152] Optionally, the recognition unit 102 includes: a first recognition module, used to perform speech recognition on the speech segment to obtain speech text when the target routing channel is an intent recognition channel; a first calculation module, used to calculate the similarity between the speech text and each first preset keyword in a first preset keyword set, and determine the intent category as a first preset intent when there is a similarity greater than or equal to a first preset similarity threshold; a second calculation module, used to calculate the similarity between the speech text and each second preset keyword in a second preset keyword set, and determine the intent category as a second preset intent when there is a similarity greater than or equal to a first preset similarity threshold; and a first determination module, used to determine the intent category as a third preset intent when there is no similarity greater than or equal to a first preset similarity threshold in either the first preset keyword set or the second preset keyword set.

[0153] Optionally, the voice command scheduling device further includes: a first monitoring module, used to monitor the number of times the third preset intent appears within a preset time period after the intent category is determined to be the third preset intent; a first generation module, used to generate prompt words and monitor the passenger's response voice when the number of occurrences is greater than or equal to a preset number threshold and other voice segments are detected, wherein the prompt words are used to guide the passenger to respond in order to obtain the response voice; and a first modification module, used to re-identify the response intent category of the response voice and modify the response intent category to the first preset intent or the second preset intent when the response intent category is the third preset intent.

[0154] Optionally, the recognition unit 102 further includes: a second recognition module, used to perform speech recognition on speech segments to obtain speech text when the target routing channel is a speech processing channel; a second construction module, used to construct a first function, a second function, and a third function, wherein the first function is used to determine the coherence of the speech segments, the second function is used to determine whether the speech segment and the previous speech segment are from the same speaker, and the third function is used to determine whether the target words in the speech text have referents in the historical discourse of a preset-length historical discourse sliding window, wherein the target words are pronouns and / or indicator words, and the preset length is used to indicate the number of historical discourses in the historical discourse sliding window; a second determination module, used to process the speech text based on the first function, the second function, and the third function to determine the target speech text; and a third recognition module, used to perform intent recognition on the target speech text to obtain the intent category, wherein the intent recognition includes: a first-level recognition and a second-level recognition, wherein if the confidence level of the first-level recognition is less than the confidence threshold, the second-level recognition is entered to determine the intent category.

[0155] Optionally, the second construction module includes: a first calculation submodule, used to calculate the semantic similarity between the speech text and each historical discourse in the historical discourse sliding window; a first construction submodule, used to construct a semantic similarity weight for each semantic similarity based on a preset attenuation coefficient, a preset length, and the position of the historical discourse corresponding to the semantic similarity in the historical discourse sliding window; and a second construction submodule, used to construct a first function based on all semantic similarities and all semantic similarity weights.

[0156] Optionally, each speech segment corresponds to a generation timestamp, and each historical utterance corresponds to a historical generation timestamp. The second construction module further includes: a first determination submodule, used to determine the target historical utterance based on the generation timestamp and all historical generation timestamps, wherein the target historical utterance is the historical utterance closest to the generation timestamp of the speech text in the historical utterance sliding window; a first extraction submodule, used to extract the first voiceprint feature vector of the speech segment corresponding to the speech text, and extract the second voiceprint feature vector of the historical speech segment corresponding to the target historical utterance; a third construction submodule, used to construct a vector similarity calculation function between the first voiceprint feature vector and the second voiceprint feature vector, and construct a second function based on the vector similarity calculation function and a preset similarity threshold; a second extraction submodule, used to extract the target word set in the speech text, and construct a target word reference set, wherein the target word reference set is used to store the words referred to by the target words found in the historical utterance sliding window; and a fourth construction submodule, used to construct a third function based on the target word set and the target word reference set.

[0157] The aforementioned voice command scheduling device may also include a processor and a memory. The aforementioned acquisition unit 100, routing unit 101, recognition unit 102, judgment unit 103, scheduling unit 104, etc., are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0158] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the scheduling priority of each historical voice command task in the task queue is determined by adjusting kernel parameters. Based on the scheduling priorities of all historical voice command tasks, the voice command tasks are scheduled.

[0159] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0160] It should be noted that the acquisition unit 100, routing unit 101, identification unit 102, determination unit 103, and scheduling unit 104 mentioned above correspond to steps S201 to S205 in Embodiment 1. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above units can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0161] Example 3

[0162] Embodiments of this application may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the aforementioned computer terminal may also be replaced with a mobile terminal or an electronic device, etc.

[0163] Optionally, in this embodiment, the computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0164] In this embodiment, the computer terminal can execute the program code for the following steps in the voice command scheduling method: acquiring the audio stream of passengers monitored by the self-service baggage check-in equipment and acquiring the current locking status of the business operation interface of the self-service baggage check-in equipment; performing voice activity detection on the audio stream to obtain voice segments, and routing the voice segments based on a preset routing function to determine the target routing channel, wherein the preset routing function is used to control the voice segments to enter the target routing channel based on the current locking status; recognizing the voice segments based on the target routing channel to obtain the intent category, and creating a voice command task based on the current locking status and the intent category. The process involves several steps: First, each voice command task has a corresponding task status. Second, each historical voice command task in the task queue is assessed for failure. Based on this assessment, the historical voice command task is replaced with a new voice command task. Third, the task queue stores a set of tasks selected from the task registry. This set includes voice command tasks in both the created and queued states. The task registry stores multiple historical voice command tasks that have been created. Finally, the scheduling priority of each historical voice command task in the task queue is determined, and the voice command task is scheduled based on the scheduling priorities of all historical voice command tasks.

[0165] Optionally, the computer terminal described above can execute the program code for the following steps in the voice command scheduling method: constructing a finite state automaton, wherein the finite state automaton includes: a locked state set, an event set, a state transition function, an initial locked state, and a mapping function. The locked state set includes: an idle state, a soft locked state, and a hard locked state. The event set includes: multiple business events triggered by passengers performing business operations on self-service baggage check-in equipment. The state transition function is used to transition the current locked state based on the business events in the event set.

[0166] Optionally, the computer terminal can execute the following steps of the voice command scheduling method: when the current locking state of the self-service baggage check-in device is a hard lock state, determine the duration of the current locking state being in the hard lock state; if the duration is longer than the preset allowed duration, convert the hard lock state to a soft lock state and roll back any unfinished business events.

[0167] Optionally, the computer terminal described above can execute the following steps in the voice command scheduling method: when the target routing channel is an intent recognition channel, perform speech recognition on the speech segment to obtain speech text; calculate the similarity between the speech text and each first preset keyword in the first preset keyword set, and determine the intent category as a first preset intent if there is a similarity greater than or equal to a first preset similarity threshold; calculate the similarity between the speech text and each second preset keyword in the second preset keyword set, and determine the intent category as a second preset intent if there is a similarity greater than or equal to a first preset similarity threshold; and determine the intent category as a third preset intent if there is no similarity greater than or equal to a first preset similarity threshold in either the first preset keyword set or the second preset keyword set.

[0168] Optionally, the computer terminal described above can execute the program code for the following steps in the voice command scheduling method: when the intent category is a third preset intent, monitor the number of times the third preset intent appears within a preset time period; when the number of occurrences is greater than or equal to a preset threshold and other voice segments are detected, generate prompt words and monitor the passenger's response voice, wherein the prompt words are used to guide the passenger to respond in order to obtain a response voice; re-identify the response intent category of the response voice, and when the response intent category is a third preset intent, modify the response intent category to a first preset intent, or modify the response intent category to a second preset intent.

[0169] Optionally, the computer terminal described above can execute the following steps of the voice command scheduling method: when the target routing channel is a voice processing channel, perform voice recognition on the voice segment to obtain voice text; construct a first function, and construct a second function and a third function, wherein the first function is used to determine the coherence of the voice segment, the second function is used to determine whether the voice segment and the previous voice segment are from the same speaker, and the third function is used to determine whether the target word in the voice text has a referent in the historical discourse of a preset-length historical discourse sliding window, wherein the target word is a pronoun and / or an indicator word, and the preset length is used to indicate the number of historical discourses in the historical discourse sliding window; process the voice text based on the first function, the second function, and the third function to determine the target voice text; perform intent recognition on the target voice text to obtain the intent category, wherein the intent recognition includes: a first-level recognition and a second-level recognition, and if the confidence of the first-level recognition is less than the confidence threshold, proceed to the second-level recognition to determine the intent category.

[0170] Optionally, the computer terminal described above can execute the following steps of the voice command scheduling method: calculating the semantic similarity between the voice text and each historical utterance in the historical utterance sliding window; for each semantic similarity, constructing a semantic similarity weight based on a preset attenuation coefficient, a preset length, and the position of the historical utterance corresponding to the semantic similarity in the historical utterance sliding window; and constructing a first function based on all semantic similarities and all semantic similarity weights.

[0171] Optionally, the computer terminal described above can execute the following steps in the voice command scheduling method: Based on the generation timestamp and all historical generation timestamps, determine the target historical utterance, wherein the target historical utterance is the historical utterance closest to the generation timestamp of the speech text in the historical utterance sliding window; extract the first voiceprint feature vector of the speech segment corresponding to the speech text, and extract the second voiceprint feature vector of the historical speech segment corresponding to the target historical utterance; construct a vector similarity calculation function between the first and second voiceprint feature vectors, and construct a second function based on the vector similarity calculation function and a preset similarity threshold; extract the target word set from the speech text, and construct a target word reference set, wherein the target word reference set is used to store the words referred to by the target words found in the historical utterance sliding window; construct a third function based on the target word set and the target word reference set.

[0172] Optionally, Figure 11 This is a structural block diagram of an electronic device according to an embodiment of this application. Figure 11 As shown, the electronic device may include: one or more ( Figure 11 (Only one is shown) Processor 1102, memory 1104, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0173] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the voice command scheduling method and apparatus in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the aforementioned voice command scheduling method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0174] The processor can access the information and application programs stored in the memory via the transmission device to execute the steps described above in the voice command scheduling method.

[0175] The embodiments of this application provide a scheme for scheduling voice commands. Through a multi-level service locking mechanism based on finite state automata, pre-routing filtering, dual-channel causal consistency guarantee based on vector clock, and speech coherence filtering, the voice processing channel can be intelligently switched according to the current service status, the task lifecycle can be dynamically managed, and the synchronization of the front-end and back-end states can be ensured. It effectively distinguishes between the main passenger's intention and interference in multi-person sharing scenarios, and realizes deep collaboration between voice commands and business operations, precise scheduling of concurrent commands, and security guarantee of critical business operations. This solves the technical problem in related technologies that cannot achieve concurrent control and collaboration between voice commands and business operations, resulting in the inability to accurately schedule voice commands for voice task processing according to the current service operation status.

[0176] Those skilled in the art will understand that Figure 11 The structure shown is for illustrative purposes only. Electronic devices can also be terminal devices such as smartphones, tablets, PDAs, and mobile internet devices (MIDs). Figure 11 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 11 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 11 The different configurations shown.

[0177] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0178] Example 4

[0179] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the voice command scheduling method provided in Embodiment 1.

[0180] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0181] This application also provides a computer program product, which, when executed on a data processing device, is a program adapted to perform scheduling method steps for voice commands.

[0182] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0183] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0184] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0185] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0186] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0187] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0188] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method of scheduling voice commands, the method comprising: include: Acquire the audio stream of passengers monitored by the self-service baggage check-in device, and obtain the current locked status of the business operation interface of the self-service baggage check-in device; The audio stream is subjected to voice activity detection to obtain voice segments, and the voice segments are routed based on a preset routing function to determine the target routing channel. The preset routing function is used to control the voice segments to enter the target routing channel based on the current locking state. Based on the target routing channel, the voice segment is identified to obtain the intent category, and a voice command task is created based on the current locking state and the intent category, wherein the voice command task corresponds to a task state; Each historical voice command task in the task queue to be scheduled is invalidated, and a determination result is obtained. Based on the determination result, the historical voice command task is replaced with the voice command task. The task queue to be scheduled stores a set of tasks to be scheduled selected from the task registry. The set of tasks to be scheduled includes the voice command tasks whose task status is either created or queued. The task registry stores multiple historical voice command tasks that have been created. Determine the scheduling priority of each of the historical voice command tasks in the queue of tasks to be scheduled, and schedule the voice command tasks based on the scheduling priorities of all the historical voice command tasks.

2. The method of scheduling voice commands of claim 1, wherein, Before obtaining the current locked state of the business operation interface of the self-service baggage check-in device, the following steps are included: A finite state automaton is constructed, wherein the finite state automaton includes: a locked state set, an event set, a state transition function, an initial locked state, and a mapping function. The locked state set includes: an idle state, a soft locked state, and a hard locked state. The event set includes: multiple business events triggered by the passenger performing business operations on the self-service baggage check-in device. The state transition function is used to transition the current locked state based on the business events in the event set.

3. The method of scheduling voice commands of claim 1, wherein, After obtaining the current locked state of the business operation interface of the self-service baggage check-in device, the following is included: If the current locking state of the self-service baggage check-in device is a hard lock state, determine the duration of the current locking state being in the hard lock state; If the duration exceeds the preset allowed duration, the hard lock state is converted to a soft lock state, and unfinished business events are rolled back.

4. The method of scheduling voice commands of claim 1, wherein, The step of identifying the speech segment and obtaining the intent category based on the target routing channel includes: If the target routing channel is an intent recognition channel, the speech segment is subjected to speech recognition to obtain speech text; Calculate the similarity between the voice text and each first preset keyword in the first preset keyword set, and determine the intent category as the first preset intent if there is a similarity greater than or equal to the first preset similarity threshold; Calculate the similarity between the voice text and each second preset keyword in the second preset keyword set, and if there is a similarity greater than or equal to the first preset similarity threshold, determine the intent category as the second preset intent; If there is no similarity greater than or equal to the first preset similarity threshold in either the first preset keyword set or the second preset keyword set, the intent category is determined as the third preset intent.

5. The method of scheduling voice commands of claim 4, wherein, After determining the intent category as a third preset intent, the following is included: When the intent category is the third preset intent, monitor the number of times the third preset intent occurs within a preset time period; If the number of occurrences is greater than or equal to a preset threshold and other speech segments are detected, a prompt word is generated, and the passenger's response speech is monitored. The prompt word is used to guide the passenger to respond in order to obtain the response speech. The response intent category of the response voice is re-identified, and if the response intent category is the third preset intent, the response intent category is modified to the first preset intent, or the response intent category is modified to the second preset intent.

6. The method of scheduling voice commands of claim 1, wherein, The step of identifying the speech segment based on the target routing channel to obtain the intent category further includes: If the target routing channel is a voice processing channel, the voice segment is subjected to voice recognition to obtain voice text; A first function is constructed, and a second and a third function are also constructed. The first function is used to determine the coherence of the speech segment, the second function is used to determine whether the speech segment and the previous speech segment are from the same speaker, and the third function is used to determine whether the target word in the speech text has a referent in the historical discourse of a preset-length historical discourse sliding window. The target word is a pronoun and / or an indicator word. The preset length is used to indicate the number of historical discourses in the historical discourse sliding window. Based on the first function, the second function, and the third function, the speech text is processed to determine the target speech text; The target speech text is subjected to intent recognition to obtain the intent category. The intent recognition includes: a first-level recognition and a second-level recognition. If the confidence level of the first-level recognition is less than a confidence threshold, the second-level recognition is performed to determine the intent category.

7. The method of scheduling voice commands of claim 6, wherein, The steps to construct the first function include: Calculate the semantic similarity between the spoken text and each of the historical discourses in the historical discourse sliding window; For each semantic similarity, a semantic similarity weight is constructed based on a preset attenuation coefficient, the preset length, and the position of the historical discourse corresponding to the semantic similarity in the historical discourse sliding window. The first function is constructed based on all the semantic similarities and all the semantic similarity weights.

8. The method of scheduling voice commands of claim 6, wherein, The speech segment corresponds to a generation timestamp, and each historical utterance corresponds to a historical generation timestamp. The steps for constructing the second and third functions include: Based on the generation timestamp and all the historical generation timestamps, a target historical discourse is determined, wherein the target historical discourse is the historical discourse in the historical discourse sliding window that is closest to the generation timestamp of the speech text. Extract the first voiceprint feature vector of the speech segment corresponding to the speech text, and extract the second voiceprint feature vector of the historical speech segment corresponding to the target historical discourse; Construct a vector similarity calculation function between the first voiceprint feature vector and the second voiceprint feature vector, and construct the second function based on the vector similarity calculation function and a preset similarity threshold; Extract the target word set from the speech text and construct the target word reference set, wherein the target word reference set is used to store the words referred to by the target words found in the historical speech sliding window; The third function is constructed based on the target word set and the target word reference set.

9. A voice command scheduling apparatus, characterized by comprising: include: The acquisition unit is used to acquire the audio stream of passengers monitored by the self-service baggage check-in device, and to acquire the current locked status of the business operation interface of the self-service baggage check-in device. The routing unit is used to perform voice activity detection on the audio stream, obtain voice segments, and route the voice segments based on a preset routing function to determine a target routing channel. The preset routing function is used to control the voice segments to enter the target routing channel based on the current locking state. The recognition unit is used to recognize the voice segment based on the target routing channel, obtain the intent category, and create a voice command task based on the current locking state and the intent category, wherein the voice command task corresponds to a task state; The determination unit is used to determine the failure of each historical voice command task in the task queue to be scheduled, obtain the determination result, and replace the historical voice command task with the voice command task based on the determination result. The task queue to be scheduled stores a set of tasks to be scheduled selected from the task registry. The set of tasks to be scheduled includes: the voice command tasks whose task status is created and queued. The task registry stores multiple historical voice command tasks that have been created. The scheduling unit is used to determine the scheduling priority of each of the historical voice command tasks in the task queue to be scheduled, and to schedule the voice command tasks based on the scheduling priorities of all the historical voice command tasks.

10. A computer program product, characterised in that, The method includes a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the voice command scheduling method according to any one of claims 1 to 8.

11. An electronic device, comprising: It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the voice command scheduling method according to any one of claims 1 to 8.

Images