Anti-noise elevator voice instruction recognition and triggering method and device and storage medium

Abstract

The application provides an anti-noise elevator voice instruction recognition and triggering method, device and storage medium, and belongs to the elevator technical field.The method comprises the following steps: acquiring an input audio stream, and performing voice activity detection on the audio stream to obtain an acoustic feature confidence; performing automatic speech recognition on the audio stream to obtain an incremental text segment, and calculating a semantic feature confidence of the incremental text segment; dynamically determining a fusion weight according to the environmental noise intensity, weighting and fusing the acoustic feature confidence and the semantic feature confidence to obtain a fusion confidence; storing the incremental text segment with the fusion confidence higher than a preset threshold in a cache; and triggering the generation of an elevator voice instruction according to the incremental text segment in the cache.The application fuses acoustic and semantic features by weighting, dynamically adapts to the environmental noise intensity, discards noise segments with low confidence, reduces false triggering from the source, and improves the reliability of elevator voice instruction recognition.

Description

Technical Field

[0001] This invention belongs to the field of elevator technology, and in particular relates to a noise-resistant elevator voice command recognition and triggering method, device and storage medium. Background Technology

[0002] In elevator voice interaction systems, accurate recognition and reliable triggering of user commands are fundamental to system reliability. However, in the elevator car or hall environment, multiple sources of interference, such as elevator operating noise, ventilation noise, conversations among multiple people, advertising announcements, and echo reverberation, pose serious challenges to voice recognition systems.

[0003] In existing technologies, acoustic feature-based voice activity detection (VAD) is typically used to detect voice endpoints in audio streams. When VAD determines that the audio is speech, it is sent to an automatic speech recognition (ASR) module for transcription, and subsequent command recognition is triggered based on the length or duration of the transcribed text. However, acoustic VAD relies solely on acoustic features for speech / non-speech decision-making. In noisy elevator environments, high-intensity noise or conversations can easily be misinterpreted as speech, resulting in ASR outputting a large amount of noisy transcribed text. This text, once cached, may trigger unnecessary elevator voice commands, causing frequent false triggers and impacting system reliability and user experience.

[0004] Therefore, how to effectively suppress noise transcription and reduce false triggering in noisy elevator environments is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a noise-resistant elevator voice command recognition and triggering method, apparatus, device, and storage medium.

[0006] The first aspect of the present invention provides a noise-resistant elevator voice command recognition and triggering method, comprising:

[0007] The input audio stream is acquired, and speech activity is detected on the audio stream to obtain acoustic feature confidence scores.

[0008] The audio stream is subjected to automatic speech recognition according to a preset transcription window to obtain incremental text segments, and the semantic feature confidence of the incremental text segments is calculated.

[0009] The fusion weights are dynamically determined based on the environmental noise intensity, and the acoustic feature confidence scores and semantic feature confidence scores are weighted and fused to obtain the fusion confidence score.

[0010] The incremental text segments with a fusion confidence level not lower than a preset threshold are stored in the cache;

[0011] Elevator voice commands are generated based on incremental text fragments in the cache.

[0012] A second aspect of the present invention provides a noise-resistant elevator voice command recognition and triggering device, comprising:

[0013] The acquisition module is used to acquire the input audio stream and perform speech activity detection on the audio stream to obtain acoustic feature confidence.

[0014] The recognition module is used to automatically recognize the audio stream according to a preset transcription window, obtain incremental text segments, and calculate the semantic feature confidence of the incremental text segments;

[0015] The fusion module is used to dynamically determine the fusion weights based on the ambient noise intensity, and to weight and fuse the acoustic feature confidence scores with the semantic feature confidence scores to obtain the fusion confidence score.

[0016] The filtering module is used to store incremental text fragments with a fusion confidence level not lower than a preset threshold into a cache;

[0017] The triggering module is used to generate elevator voice commands based on the incremental text fragments in the cache.

[0018] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the noise-resistant elevator voice command recognition and triggering method as described in the first aspect above.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the noise-resistant elevator voice command recognition and triggering method as described in the first aspect above.

[0020] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0021] This invention introduces semantic feature confidence on top of acoustic feature confidence. Acoustic feature confidence reflects whether the audio contains human voice; semantic feature confidence determines whether the incremental text fragment has true semantic meaning. After the two are fused, pure noise or conversations from others are excluded because their semantic confidence is low and the fusion confidence is unlikely to reach the threshold, thus not participating in subsequent triggering. The fusion weight is dynamically adjusted according to the intensity of ambient noise; the higher the noise, the higher the proportion of semantic features, enabling the system to maintain stable discrimination under different noise levels. Therefore, only text containing truly effective semantics is stored in the cache, reducing false triggers caused by noise transcription at the source and improving the reliability of the elevator voice interaction system in noisy environments. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of a noise-resistant elevator voice command recognition and triggering method provided in an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of another noise-resistant elevator voice command recognition and triggering method provided in an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of a noise-resistant elevator voice command recognition and triggering device provided in an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the present invention. However, those skilled in the art will recognize that the present application may be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted to avoid unnecessary detail that could obscure the description of the present application.

[0028] The technical solution of the present invention will be illustrated below through specific embodiments.

[0029] Reference Figure 1 This diagram illustrates a noise-resistant elevator voice command recognition and triggering method according to an embodiment of the present invention. This method can be executed by a noise-resistant elevator voice command recognition and triggering device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method may specifically include the following steps:

[0030] S101. Obtain the input audio stream and perform speech activity detection on the audio stream to obtain acoustic feature confidence.

[0031] In the elevator car or lobby environment, sound acquisition devices (such as microphones) continuously collect user voice and ambient sounds to form a continuous audio stream. This audio stream contains various sound components such as user commands, elevator operating noise, ventilation noise, conversations between multiple people, and advertising announcements.

[0032] Speech Activity Detection (VAD) is performed on the audio stream input from the sound acquisition device to obtain acoustic feature confidence scores. Unlike binarized VAD (which only outputs "speech present" or "no speech present"), this step outputs a continuous value to characterize the probability that the current audio segment contains human voice. The confidence score typically ranges from 0 to 1; a higher value indicates a stronger likelihood of speech, while a lower value indicates a stronger likelihood of noise.

[0033] In this embodiment, the acoustic feature confidence level can be obtained in several ways. A common approach is to use a neural network-based VAD model, which takes acoustic features such as Mel-frequency cepstral coefficients (MFCC) and spectral centroid of the audio frame as input and outputs the speech probability of the current frame. Optionally, for a continuous audio stream (e.g., every 200 milliseconds), the quantile (e.g., the 0.7 quantile) of the speech probability of all frames within the audio stream can be taken as the acoustic feature confidence level of the audio stream, which can suppress interference from individual abnormal frames.

[0034] S102. Perform automatic speech recognition on the audio stream according to the preset transcription window to obtain incremental text segments, and calculate the semantic feature confidence of the incremental text segments.

[0035] While performing speech activity detection in S101, this step performs Automatic Speech Recognition (ASR) on the same audio stream to obtain incremental text segments. An incremental text segment refers to the recognition result output by ASR in a streaming manner, with each output incremental text segment corresponding to a continuous audio window (e.g., the most recent 3 seconds of audio). As the audio stream continues to input, ASR continuously outputs new incremental text segments.

[0036] For each incremental text segment, its semantic feature confidence score is calculated. The semantic feature confidence score is used to characterize whether the text segment has true semantic content.

[0037] In elevator scenarios, pure noise or casual conversations, after being transcribed by ASR, often exhibit low semantic density features, for example:

[0038] Noise transcriptions commonly found in field logs are usually not completely blank, but fall into the following categories:

[0039] 1. Repeated words or repeated use of a single character, such as "ah ah ah ah", "um um um", "hahaha".

[0040] 2. Random English / number fragments (such as "aaa", "1 1 1", "ok ok"), but do not constitute comprehensible semantics.

[0041] 3. Short segments fragmented by echoes or broadcasts (such as “to…to…”, “open…open…”), which are short and unstable in length.

[0042] Although these texts are output by ASR, they do not contain valid elevator instruction information. Semantic feature confidence is the metric used to quantify this semantic validity.

[0043] Semantic feature confidence can be calculated in several ways. For example, a lightweight language model can be used to calculate the perplexity or entropy of the text. The smaller the entropy value, the higher the certainty of the text and the more natural it resembles a sentence, resulting in a higher semantic confidence. Alternatively, a rule-based lightweight scoring method can be used, such as counting the number of Chinese characters in the text, its repetition rate, and whether it contains elevator entity words (such as floor, open door, closed door), to obtain a confidence value between 0 and 1.

[0044] For example, the token sequence {w} of the incremental text fragment x j}, using a lightweight language model, obtain the conditional probability distribution p for each token. j (⋅), calculate the average entropy:

[0045] ;

[0046] Where m is the total number of tokens (words or subwords) in text x, v is a candidate token in the vocabulary, and j is the index of the token.

[0047] Then, the entropy is normalized to a confidence level. The smaller the entropy, the more certain and more likely it is to be a valid statement.

[0048] S103. Dynamically determine the fusion weights based on the environmental noise intensity, and weight the acoustic feature confidence and semantic feature confidence to obtain the fusion confidence.

[0049] In an elevator environment, noise intensity is not constant. Noise levels change significantly when the elevator is running, when the doors are opening and closing, and when advertisements are being broadcast. For example, the noise floor can be estimated using the energy of a non-speech frame, and then the ratio of this to the energy of the current segment can be used to obtain an approximate SNR, which can then be mapped to noise intensity.

[0050] Calculate the root mean square energy (RMS) for each frame of the audio:

[0051] ;

[0052] The energy is the energy of a single frame, where n is the frame length (number of sampling points); f is the index of the audio frame; k is the index of the sampling point within the frame, and x...f (k) represents the amplitude of the k-th sampling point in the f-th frame.

[0053] The set of frames F that are determined to be non-speech by acoustic VAD non The noise level is estimated to be low;

[0054] ;

[0055] This represents median operations. This is the estimated baseline for environmental noise energy.

[0056] Average energy of the current incremental text fragment Calculate the signal-to-noise ratio (in dB):

[0057] ;

[0058] Noise intensity is defined as .

[0059] in, It is an empirical benchmark (e.g., 40dB). To prevent abnormal values, the upper limit of noise intensity is set at 80dB. Pick Prevent division by zero. The average energy of the current incremental text segment. It is a very small positive number.

[0060] When the ambient noise is weak, the acoustic VAD is more reliable and can be given a higher weight for confidence of acoustic features. When the ambient noise is strong, acoustic features are easily interfered with. In this case, the weight of acoustic confidence should be reduced and more reliance should be placed on the confidence of semantic features for judgment.

[0061] In an optional embodiment, the fusion weight is the weight of the acoustic feature confidence, which is inversely proportional to the ambient noise intensity, and the sum of the weights of the acoustic feature confidence and the semantic feature confidence is 1.

[0062] For example, the weight coefficient α for the confidence of acoustic features is (1-α), while the weight for the confidence of semantic features is (1-α).

[0063] Fusion confidence Cfu = α × Ca + (1 - α) × Cs;

[0064] Where Ca is the acoustic feature confidence level and Cs is the semantic feature confidence level. The value of α is usually limited to between 0.4 and 0.6 to ensure that neither confidence level is completely ignored. Specifically, as the noise intensity increases, α decreases accordingly, making the fusion result more dependent on the semantic features.

[0065] S104. Store incremental text fragments with a fusion confidence level not lower than a preset threshold into the cache.

[0066] When the fusion confidence level is lower than a preset threshold, the current incremental text fragment is determined to be noise and is not stored in the cache, thus blocking the possibility of noisy text entering the subsequent triggering process and effectively reducing false triggering caused by environmental noise or conversations.

[0067] When the fusion confidence level is not lower than a preset threshold, the current incremental text fragment is determined to have high credibility and is stored in the cache. The cache is a temporary storage area used to accumulate filtered incremental text fragments, providing a data basis for subsequent trigger judgments.

[0068] The preset threshold is a configurable parameter used to control the strictness of noise detection. This threshold can be calibrated according to the actual scenario; for example, a conservative value (such as 0.7) can be preset, and then calibrated using pure noise segments and real command segments collected on-site. The goal is to maximize the recall rate of real commands while ensuring that the false trigger rate is below the expected upper limit.

[0069] S105. Trigger the generation of elevator voice commands based on the incremental text fragments in the cache.

[0070] The meaning of triggering generation here is that, under certain conditions, the text fragment in the cache is output as a complete elevator voice command and sent to the subsequent command recognition or dialogue understanding module.

[0071] It should be noted that the generation based on incremental text fragments in the cache in this step does not mean that it will be triggered immediately every time a fragment is stored. Rather, it means that the text fragments in the cache serve as the data source for triggering the decision. The specific triggering conditions (such as the cumulative duration reaching a threshold, the cumulative text information reaching a threshold, etc.) can be limited according to actual needs.

[0072] In elevator interaction scenarios, users' elevator voice commands often consist of multiple incremental segments. For example, if a user says "to the third floor," the ASR (Automatic Response System) might output multiple incremental segments such as "to" "third floor" or "to the third floor" and "floor." By accumulating these segments in a cache, the complete command text can be obtained, enabling accurate triggering decisions.

[0073] This step completes the entire closed loop from audio input to elevator voice command triggering: the audio stream goes through acoustic detection, speech recognition, semantic analysis, weighted fusion, filtering and caching, and finally forms a usable command text in the cache, which triggers the generation of elevator voice commands for the elevator system to perform corresponding operations (such as registering floors, opening and closing doors, etc.).

[0074] This invention introduces semantic feature confidence on top of acoustic feature confidence. Acoustic feature confidence reflects whether the audio contains human voice; semantic feature confidence determines whether the incremental text fragment has true semantic meaning. After the two are fused, pure noise or conversations from others are excluded because their semantic confidence is low and the fusion confidence is unlikely to reach the threshold, thus not participating in subsequent triggering. The fusion weight is dynamically adjusted according to the intensity of ambient noise; the higher the noise level, the higher the proportion of semantic features, enabling the system to maintain stable judgment under different noise levels. Therefore, only text containing truly effective semantics is stored in the cache, reducing false triggers caused by noise transcription at the source and improving the reliability of the elevator voice interaction system in noisy environments.

[0075] In an optional embodiment, before triggering the generation of elevator voice commands based on incremental text fragments in the cache, the method further includes:

[0076] Incremental text fragments in the cache are pruned based on their timestamps, retaining only those within the most recent preset time window. A weighted total length is calculated based on the character length of each incremental text fragment in the cache and its corresponding retention weight. When the weighted total length exceeds a preset upper limit, the incremental text fragment with the lowest retention weight in the cache is deleted. The retention weight decreases as the incremental text fragment is stored in the cache longer.

[0077] The process from a user entering the elevator car to issuing a command typically takes only a few tens of seconds. Historical text that exceeds this time window is no longer relevant to the current interaction and should not be included in the trigger judgment. For example, a time window can be set to 60 seconds, meaning only segments stored within 60 seconds are retained, while segments older than 60 seconds are discarded. This example is based on the typical cycle of elevator interactions.

[0078] Building upon this, a weighted constraint mechanism based on retention weights is further introduced. Each incremental text fragment is assigned a retention weight, which decreases as the fragment is stored in the cache longer; that is, fragments stored earlier have lower weights, and fragments stored later have higher weights.

[0079] For example, the retention weights can be represented using an exponential decay function, such as:

[0080] ;

[0081] t is the current timestamp, and t0 is the timestamp when the incremental text fragment is written to the cache.

[0082] The weighted total length is calculated based on the retention weight of each segment, which is the sum of the products of the actual number of characters in each segment and its retention weight. For example, if there are three segments in the cache with actual character counts of 3, 2, and 1, and retention weights of 0.9, 0.5, and 0.2, respectively, then the weighted total length is: 3 × 0.9 + 2 × 0.5 + 1 × 0.2 = 3.9.

[0083] The preset weighted length upper limit is used to constrain the total weighted length of the cache. When the total weighted length exceeds the upper limit, the incremental text fragment with the lowest weight is deleted from the cache until the total weighted length does not exceed the upper limit. When there is too much cached content, fragments with low weight (i.e., those stored earlier) are prioritized for deletion, while fragments with high weight (i.e., those stored later) are retained.

[0084] Through the combination of time-sensitive pruning and weighted constraints, the influence of historical incremental text fragments on subsequent trigger determination naturally decays over time. Newer text fragments have higher weight and are more likely to be retained, while older text fragments have lower weight and are preferentially eliminated when the cache is overloaded. This allows the system to focus on valid elevator voice commands in the current interaction cycle, avoiding the problem of delayed false triggers caused by excessive accumulation of historical text in the cache due to prolonged noisy environments.

[0085] In an optional embodiment, generating elevator voice commands is triggered based on incremental text fragments in the cache, including:

[0086] Calculate the cumulative speech duration and cumulative text information content corresponding to the incremental text segments in the cache; when the cumulative speech duration reaches the first duration threshold and the cumulative text information content reaches the preset information content threshold, trigger the generation of elevator voice commands based on the incremental text segments in the cache.

[0087] First, calculate the cumulative speech duration and cumulative text information content corresponding to all incremental text segments in the cache. The cumulative speech duration is the cumulative duration of valid speech frames, meaning only the time length corresponding to audio frames identified as speech by VAD is counted. This design avoids inflated durations due to user pauses or quiet environments. The cumulative text information content refers to the number of valid characters in all incremental text segments in the cache, typically counting Chinese characters while excluding punctuation, spaces, and duplicate characters.

[0088] A preset first duration threshold and an information content threshold are used as the criteria for regular triggering. The first duration threshold can be set based on the voice duration of typical commands in elevator scenarios, for example, 3 seconds; the information content threshold can be set based on the minimum number of valid characters in the command, for example, 10 Chinese characters. When the cumulative voice duration reaches the first duration threshold and the cumulative text information content reaches the information content threshold, it is determined that the user has fully expressed their intent, and elevator voice commands are triggered based on the incremental text fragments in the cache.

[0089] Relying solely on duration might lead to false triggers due to intermittent user speech, while relying solely on character count might result in missed triggers due to sparse text. Combining both ensures that the command is only triggered when the user speaks for a sufficient duration and generates enough information, effectively balancing timeliness and reliability. For example, if a user says "to the third floor," the entire pronunciation lasts approximately 2 seconds, accumulating 3 characters, which is below the threshold and continues to accumulate. When the user continues saying "open the door" or repeating "to the third floor," the duration and character count gradually accumulate until the threshold is reached, triggering the command. This avoids premature triggering due to fragmented text or intermittent speech, preventing the interaction from failing to trigger.

[0090] In an optional embodiment, the noise-resistant elevator voice command recognition and triggering method further includes:

[0091] When the cumulative voice duration reaches the second duration threshold and the cumulative text information does not reach the preset information content threshold, an elevator voice command is generated based on the incremental text fragments in the cache; wherein, the second duration threshold is greater than the first duration threshold.

[0092] A second duration threshold is preset, which is greater than the first duration threshold. For example, the first duration threshold can be set to 3 seconds, and the second duration threshold can be set to 10 seconds. When the cumulative voice duration reaches the second duration threshold, but the cumulative text information amount has not yet reached the preset information amount threshold, elevator voice commands are generated based on the incremental text segments in the cache.

[0093] During user interaction with elevator voice systems, the following situations may occur: users hesitate and pause repeatedly, resulting in a continuous increase in accumulated voice duration but insufficient effective text information; or environmental noise interference causes the ASR (Automatic Recognition Scripting) to output a large amount of low-information fragmented text, making it difficult to accumulate effective characters; or the user's commands contain many unpreset words, resulting in low semantic feature confidence, even though the user is indeed attempting to communicate with the system. In these situations, relying solely on conventional triggering conditions (both duration and information content meeting the threshold) may result in prolonged delays, causing interaction freezes and impacting user experience. Therefore, a forced trigger is implemented when the accumulated voice duration reaches the second duration threshold, sending the currently cached text fragments to the subsequent command recognition module for further processing.

[0094] Through the above mechanism, regular triggering (meeting both duration and information content requirements) and fallback triggering (long-term forced triggering) complement each other: the former ensures timely response when the user expresses instructions normally, while the latter ensures that the system does not freeze when the user expresses instructions intermittently or the text is sparse, thus balancing the accuracy and accessibility of triggering in elevator voice interaction in noisy environments.

[0095] In an optional embodiment, generating elevator voice commands is triggered based on incremental text fragments in the cache, including:

[0096] Gating classification is performed on the cumulative text formed by incremental text fragments in the cache that meet the triggering conditions. The result of whether to respond, the instruction category, and the minimum elevator instruction phrase extracted from the cumulative text are output. When the result is yes and the instruction category is elevator control instruction, the elevator voice instruction is generated based on the minimum elevator instruction phrase.

[0097] When an incremental text fragment in the cache meets the triggering conditions (e.g., the cumulative voice duration and cumulative text information reach a preset threshold), the system does not directly respond to the cumulative text in the cache. Instead, it first performs gating classification. The gating classifier outputs three pieces of information: the determination result of whether to respond, the instruction category, and the smallest elevator instruction phrase extracted from the cumulative text.

[0098] The "response" criterion determines whether the text pertains to elevator control. For example, phrases like "go to the third floor," "open the door," and "close the door" should be considered responses, while casual conversations like "the weather is nice today" or "it's a bit crowded" are considered non-responses. The instruction category further categorizes instructions, such as elevator control instructions, floor query instructions, and equipment status query instructions. When the criterion is "yes" and the instruction category is "elevator control instruction," the extracted minimum elevator instruction phrase is used as input to the subsequent tool planning module, triggering the generation of elevator voice instructions.

[0099] The extraction of the minimum elevator instruction phrase follows these principles: no information system is introduced, and the extraction result must be a substring of the original text or a compact rewrite of it (only irrelevant components are removed); it should be as short as possible while keeping the elevator intent unchanged; if the text contains multiple requests, only the most explicit and first-appearing one is retained; if an explicit elevator instruction phrase cannot be extracted, the extraction result is empty, and it falls back to the gating result of the entire sentence.

[0100] For example, if the ASR output text is "Please press the third floor button for me", the gate classifier extracts the shortest elevator instruction phrase "to the third floor"; if the text is "Wait a moment, I want to open the door", then it extracts "open the door"; through extraction, it can filter out modifiers that have no actual control significance and retain only the core instruction content.

[0101] In noisy elevator environments, the text output by ASR may be mixed with irrelevant content such as conversations and echoing advertisements. Directly gating entire sentences can easily lead to misjudgments of inactivity due to excessive background noise, or errors in parsing by the tool's planning module due to text redundancy. By using gating classification and short instruction extraction, core instructions can be quickly identified, thereby reducing the probability of accidental discarding and misparsing.

[0102] In an optional embodiment, the method further includes:

[0103] When the elevator voice command is triggered, the right boundary of the transcription window corresponding to the trigger time is recorded as the sample point boundary in the audio stream; sample point pruning is performed on the audio stream, and audio data located before the sample point boundary in the audio stream is discarded so that subsequent transcription is based on audio data after the sample point boundary.

[0104] In streaming automatic speech recognition (ASR) processing, a sliding window approach is typically used to continuously recognize audio streams. Each transcription window corresponds to a fixed-length audio segment (e.g., the most recent 3 seconds), and the window slides forward continuously over time, outputting an incremental text segment with each window. This mechanism ensures real-time recognition during normal interactions, but it also introduces a problem: after responding to a user's elevator voice command, subsequent transcription windows may still contain the same historical audio segment, causing the same command to be repeatedly recognized and triggered.

[0105] To address the aforementioned issues, this embodiment records the right boundary of the transcription window corresponding to the trigger moment when generating the elevator voice command, using it as the sample point boundary in the audio stream. The sample point boundary is a precise location in the audio stream, represented by a sampling point number. For example, if the sampling rate is 16000Hz, and the right boundary of the transcription window corresponding to the trigger moment is the 160000th sampling point, then this value is recorded as the sample point boundary.

[0106] Subsequently, sample point pruning is performed on the audio stream, discarding all audio data located before the boundary of that sample point. This operation directly affects the ASR's audio input buffer, essentially defining the boundary between processed and unprocessed data at the audio level. After pruning, the ASR's transcription window will slide only based on the audio data after the sample point boundary.

[0107] Optionally, after performing sample point pruning on the audio stream, a new speaking session ID is assigned to identify the different session text.

[0108] Through the above mechanism, subsequent transcription windows no longer contain any historical audio prior to the triggering time, fundamentally preventing the possibility of the same historical audio being transcribed repeatedly. Compared with traditional text similarity deduplication or time-silent period schemes, this scheme achieves deterministic round boundaries at the audio level, does not rely on threshold tuning, and has higher reliability and interpretability.

[0109] It should be noted that the recording and cropping of sample point boundaries are performed in real time, immediately after the response is triggered. After cropping, the system continues to listen for subsequent user voice input, entering a new interaction cycle, thus ensuring that there is no audio-level interference between multiple interaction rounds.

[0110] In an optional embodiment, the noise-resistant elevator voice command recognition and triggering method further includes:

[0111] Set an anti-replay barrier, the value of which is the boundary of the sample point; for subsequent arriving incremental text fragments, if the left boundary of the transcription window corresponding to the incremental text fragment is smaller than the anti-replay barrier, then the incremental text fragment is discarded.

[0112] In elevator interaction scenarios, Automatic Speech Recognition (ASR) typically uses a sliding window approach to process audio streams. The problem is that after the system has responded to a user command, subsequent sliding windows may still contain the same historical audio, causing the same command to be repeatedly transcribed and triggered.

[0113] Specifically, when the elevator voice command is triggered (i.e., when the text in the cache has been determined to be a valid command and the response is ready to be executed), the position of the right boundary of the transcription window corresponding to the trigger time in the audio stream is recorded. This position is represented by the sample point number and is denoted as the sample point boundary. This boundary is equivalent to a cutoff point, indicating that the audio before this point has been processed and should not be included in the subsequent transcription and triggering process. Regardless of how the ASR sliding window slides or whether there are historical segments in transit in the asynchronous queue, as long as the left boundary of its audio window is earlier than the right boundary of the trigger time, it will be blocked, thereby preventing repeated triggering caused by the replay of historical audio from the source.

[0114] Then, an anti-replay barrier is set, with its value set to the boundary of the aforementioned sample points. The anti-replay barrier is a threshold maintained in the text link to determine whether subsequently arriving incremental text segments originate from historical audio.

[0115] Subsequently, for each arriving incremental text segment, the left boundary of the transcription window corresponding to that segment (i.e., the starting position of the audio covered by that segment) is checked. If the left boundary is less than the value of the anti-replay barrier, it means that the audio window corresponding to that incremental text segment starts before the barrier, that is, the segment comes from previously processed historical audio, and is therefore identified as a historical replay segment and discarded directly, without entering the subsequent triggering process.

[0116] In an optional embodiment, it further includes:

[0117] When the system is in broadcast mode, cached text that meets the trigger conditions is delayed and stored in the pending-trigger buffer without being triggered immediately; when a high-priority interruption instruction is detected in the cached text, it is triggered immediately and the cached text is sent to the subsequent instruction recognition module, while a broadcast interruption indication is output; when the broadcast mode ends, the text in the pending-trigger buffer is sent to the subsequent instruction recognition module.

[0118] To clearly illustrate the process and technical effects of the noise-resistant elevator voice command recognition and triggering method of this application, the following example is used for explanation: Figure 2 The diagram illustrates another noise-resistant elevator voice command recognition and triggering method, which includes the following steps:

[0119] S1. Obtain the input audio stream and segment the audio into a continuous sequence of segments.

[0120] S2, for each audio segment a i Calculate the confidence level of the acoustic features and estimate the noise intensity, and calculate the timestamp t. i The corresponding audio segment a i Effective speech duration a i '.

[0121] S3, correspondingly obtain the ASR incremental text fragment p i , for p i Perform language-side filtering to obtain p i '; if p i If empty, skip the accumulation.

[0122] S4. Calculate the semantic confidence score and dynamically determine the weights according to the noise intensity to obtain the fusion confidence score C. fu .

[0123] S5, when C fu When the value is less than a preset threshold, output empty text or do not write to the buffer; when C fu When it is not less than the preset threshold, (t) i p i ',a i Write the corresponding speaking session ID to the realtime cache.

[0124] S6. Perform time-sensitivity pruning on the realtime cache: only retain caches that meet the time-sensitivity requirement. i Segments ≤ 60s, and ensure that the sum of weighted lengths is less than the preset upper limit of weighted length.

[0125] S7. Concatenate the realtime cache to obtain the merged text. Calculate the cumulative speech duration T and the number of valid Chinese characters L.

[0126] S8. Determine whether the event is triggered based on the preset F(T0, L0); if it is not triggered, continue accumulating.

[0127] S9. When in the middle of broadcast (during_tts=True), if x is determined to be a high-priority interrupt instruction, the interrupt indicator will be triggered and output immediately; otherwise, x will be written to pending and the trigger will be delayed.

[0128] S10. When the broadcast end event arrives, read the text to be triggered in pending and output it as exec, while clearing realtime / pending to form a round boundary.

[0129] S11. Send exec as the dialogue awareness input to the subsequent dialogue understanding / instruction recognition module to complete the input construction of the noise instruction recognition link.

[0130] S12. When determining whether to proceed to subsequent instruction recognition / tool ​​planning, record the corresponding sample point boundary S that triggers the process. cut (Take the right boundary of the transcription window for this time), and issue a pruning command to make the ASR audio cache perform sample point pruning.

[0131] You can also switch the conversation indicator to enter a new round of accumulation.

[0132] S13, Update the anti-replay barrier to S cut Subsequent transcribed fragments arriving at the left boundary of the window are discarded to ensure strict round boundaries and eliminate repeated triggering caused by asynchronous in-transit fragments.

[0133] It should be noted that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0134] Reference Figure 3 The diagram illustrates a noise-resistant elevator voice command recognition and triggering device according to an embodiment of the present invention, which may specifically include the following modules:

[0135] The acquisition module 301 is used to acquire the input audio stream and perform speech activity detection on the audio stream to obtain acoustic feature confidence.

[0136] The recognition module 302 is used to perform automatic speech recognition on the audio stream, obtain incremental text segments, and calculate the semantic feature confidence of the incremental text segments;

[0137] The fusion module 303 is used to dynamically determine the fusion weights based on the ambient noise intensity, and to weight and fuse the acoustic feature confidence scores with the semantic feature confidence scores to obtain the fusion confidence scores.

[0138] The filtering module 304 is used to store incremental text fragments with a fusion confidence level not lower than a preset threshold into a cache;

[0139] Trigger module 305 is used to trigger the generation of elevator voice commands based on the incremental text fragments in the cache.

[0140] The present invention provides a noise-resistant elevator voice command recognition and triggering device. By applying the noise-resistant elevator voice command recognition and triggering device, the various steps in the aforementioned noise-resistant elevator voice command recognition and triggering method embodiments can be realized.

[0141] It should be noted that the module division in the various noise-resistant elevator voice command recognition and triggering devices provided in the above embodiments is illustrative and only represents one logical functional division. In actual implementation, other division methods may also be used. Furthermore, the functional modules in the various embodiments of this invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0142] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the technical solution of the embodiments of the present invention can be embodied in the form of a computer program product, which is stored in a computer storage medium and includes several instructions to cause an electronic device or processor to execute all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned computer storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] Furthermore, the noise-resistant elevator voice command recognition and triggering device and the noise-resistant elevator voice command recognition and triggering method provided in the above embodiments belong to the same concept. For details of their specific implementation process, please refer to the method embodiments, which will not be repeated here.

[0144] Reference Figure 4 The diagram illustrates an electronic device according to an embodiment of the present invention. Figure 4 As shown, the electronic device in this embodiment of the invention includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the above-described embodiment of the noise-resistant elevator voice command recognition and triggering method. Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-described embodiment of the noise-resistant elevator voice command recognition and triggering device.

[0145] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which can be used to describe the execution process of the computer program in the electronic device.

[0146] The electronic device may be a desktop computer, a cloud server, or other computing device. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 4 This is merely one example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0147] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0148] The memory can be an internal storage unit of the electronic device, such as a hard drive or RAM. Alternatively, it can be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory can include both internal and external storage units. The memory is used to store the computer program and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or will be output.

[0149] This invention also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the noise-resistant elevator voice command recognition and triggering method as described in the foregoing embodiments.

[0150] This invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the noise-resistant elevator voice command recognition and triggering method as described in the foregoing embodiments.

[0151] This invention also discloses a computer program product that, when run on a computer, causes the computer to execute the noise-resistant elevator voice command recognition and triggering method described in the foregoing embodiments.

[0152] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for noise-resistant elevator voice command recognition and triggering, characterized in that, include: The input audio stream is acquired, and speech activity is detected on the audio stream to obtain acoustic feature confidence scores. The audio stream is subjected to automatic speech recognition according to a preset transcription window to obtain incremental text segments, and the semantic feature confidence of the incremental text segments is calculated. The fusion weights are dynamically determined based on the environmental noise intensity, and the acoustic feature confidence scores and semantic feature confidence scores are weighted and fused to obtain the fusion confidence score. The incremental text segments with a fusion confidence level not lower than a preset threshold are stored in the cache; Elevator voice commands are generated based on incremental text fragments in the cache.

2. The method according to claim 1, characterized in that, The fusion weight is the weight of the acoustic feature confidence level, the fusion weight is inversely proportional to the ambient noise intensity, and the sum of the weights of the acoustic feature confidence level and the semantic feature confidence level is 1.

3. The method according to claim 1, characterized in that, Before triggering the generation of elevator voice commands based on the incremental text fragments in the cache, the method further includes: The incremental text fragments in the cache are pruned according to their timestamps, retaining only the incremental text fragments within the most recent preset time window; The weighted total length is calculated based on the character length of the incremental text fragments in the cache and the corresponding retention weights; When the total weighted length exceeds the preset upper limit of the weighted length, delete the incremental text segment with the lowest retention weight in the cache; The retention weight decreases as the time the incremental text fragment is stored in the cache increases.

4. The method according to claim 1, characterized in that, The step of triggering the generation of elevator voice commands based on the incremental text fragments in the cache includes: Calculate the cumulative speech duration and cumulative text information content corresponding to the incremental text segments in the cache; When the cumulative voice duration reaches a first duration threshold and the cumulative text information reaches a preset information amount threshold, an elevator voice command is generated based on the incremental text fragments in the cache.

5. The method according to claim 4, characterized in that, Also includes: When the cumulative voice duration reaches the second duration threshold and the cumulative text information does not reach the preset information content threshold, an elevator voice command is generated based on the incremental text fragments in the cache. Wherein, the second duration threshold is greater than the first duration threshold.

6. The method according to claim 4 or 5, characterized in that, The step of triggering the generation of elevator voice commands based on the incremental text fragments in the cache includes: Gating classification is performed on the cumulative text formed by the incremental text fragments in the cache that meet the triggering conditions, and the result of whether to respond, the instruction category, and the smallest elevator instruction phrase extracted from the cumulative text is output. When the determination result is yes and the instruction category is elevator control instruction, an elevator voice instruction is generated based on the minimum elevator instruction phrase.

7. The method according to claim 1, characterized in that, Also includes: When the elevator voice command is triggered, the right boundary of the transcription window corresponding to the trigger time is recorded as the sample point boundary in the audio stream; The audio stream is pruned by discarding audio data that is located before the boundary of the sample point, so that subsequent transcription is based on the audio data after the boundary of the sample point.

8. The method according to claim 7, characterized in that, Also includes: Set an anti-replay barrier, the value of which is the boundary of the sample point; For subsequent incremental text fragments, if the left boundary of the transcription window corresponding to the incremental text fragment is smaller than the anti-replay barrier, the incremental text fragment is discarded.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the noise-resistant elevator voice command recognition and triggering method as described in any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the noise-resistant elevator voice command recognition and triggering method as described in any one of claims 1-8.

Images