Speech instruction dynamic recognition and separation method based on multi-person speech scene

Abstract

The application discloses a voice instruction dynamic recognition and separation method based on a multi-person voice scene, relates to the technical field of voice instruction recognition, and is used for solving the problem of low multi-task voice control efficiency. After a kitchen voice control device is started, ambient audio streams are listened to and divided into audio frames, frame energy and zero-crossing rate are extracted to recognize human voice segments, the fundamental frequency of the human voice segments is collected, and acoustic deviation features are calculated, instruction issuing targets are created according to the acoustic deviation, the sound duration of each target is detected to obtain effective instructions, the issuing time of the effective instructions is called, overlapping time is calculated, and parallel execution or overlapping sorting mechanism is selected based on the overlapping time. In the overlapping sorting mechanism, the operation state of the corresponding cooking device and the audio amplitude are obtained, the instruction priority level is set in combination with historical registration information, the effective instructions are sorted to generate an execution sequence, and the execution sequence is executed in sequence one by one, so that the response speed and accuracy of kitchen multi-task voice instruction processing are improved.

Description

Technical Field

[0001] This invention relates to the field of voice command recognition technology, and more specifically, to a method for dynamic recognition and separation of voice commands in multi-person voice scenarios. Background Technology

[0002] With the development of smart kitchens, voice control technology has gradually become an important interactive method for operating kitchen equipment. Existing kitchen voice control equipment mainly recognizes and executes single-person voice commands, and can complete operations such as turning cooking equipment on and off, adjusting temperature, and setting cooking modes in simple scenarios.

[0003] The existing technology has the following shortcomings:

[0004] Currently, existing technologies only recognize commands in single-person voice environments and lack a dynamic separation and sorting mechanism for voice commands issued by multiple people simultaneously. This can easily lead to command conflicts and device malfunctions, resulting in low efficiency, response delays, and a decline in user experience in multi-task voice control. Therefore, a method for dynamic recognition and separation of voice commands based on multi-person voice scenarios is proposed.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for dynamic recognition and separation of voice commands based on multi-person voice scenarios. This method addresses the problems mentioned in the background art by employing environmental audio stream frame segmentation, frame energy and zero-crossing rate analysis, acoustic deviation feature extraction, command speech duration detection, and overlapping command sorting and priority setting mechanisms.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for dynamic recognition and separation of voice commands in multi-person voice scenarios, comprising the following steps:

[0008] Step S1: After the kitchen voice control device is turned on, it listens to the surrounding environment to obtain the ambient audio stream, divides the ambient audio stream into different audio frames, obtains the frame energy and zero-crossing rate of the audio frames, and extracts the human voice segments in the ambient audio stream by combining the frame energy and zero-crossing rate.

[0009] Step S2: Collect the fundamental frequency data of the human voice segment and calculate the acoustic deviation characteristics. Create different command issuance targets based on the acoustic deviation characteristics. Detect the sound duration of the human voice segment corresponding to each command issuance target. Collect the valid commands of each command issuance target based on the sound duration.

[0010] Step S3: Retrieve the issuance time of each valid instruction, calculate the overlap time of the valid instructions based on the issuance time, and select to trigger the parallel instruction execution mechanism or the overlapping instruction sorting mechanism based on the overlap time. In the overlapping instruction sorting mechanism, retrieve the operating status value of the cooking equipment corresponding to the valid instruction.

[0011] Step S4: Detect the audio amplitude of valid commands, obtain the registration target information through the historical database, set the priority level of the command issuing target by combining the registration target information and audio amplitude, sort the valid commands according to the operation status value and generate the command execution sequence, and execute the valid commands one by one according to the command execution sequence.

[0012] In a preferred embodiment, in step S1, the kitchen voice control device enters a listening mode after being turned on, and collects the acoustic signals of the surrounding environment in real time to obtain an environmental audio stream.

[0013] Ambient audio streams are continuous analog sound signals that vary over time, including speech signals, ambient noise, and other non-speech sound source signals;

[0014] The analog sound signal is digitized, that is, the analog sound signal is sampled at a fixed sampling rate by the built-in audio sampling module to obtain a digitized discrete signal sequence.

[0015] A fast Fourier transform is performed on the digitized discrete signal sequence, and high-frequency and low-frequency components belonging to the noise region in the spectrum are initially filtered out according to the preset noise frequency band threshold rules.

[0016] In a preferred embodiment, the processed ambient audio stream is divided into multiple audio frames of equal length in chronological order, and for each audio frame, the frame energy and zero-crossing rate are calculated:

[0017] Frame energy characterizes the energy intensity of an audio frame in the time domain;

[0018] The zero-crossing rate characterizes the number of times the waveform of a digitized discrete signal crosses the zero amplitude point within an audio frame;

[0019] The frame energy and zero-crossing rate are standardized separately to obtain the frame energy factor and zero-crossing rate factor. The frame energy factor and zero-crossing rate factor are added together to obtain the human voice evaluation value.

[0020] When the human voice evaluation value is greater than or equal to the preset human voice threshold, the audio frame is determined to be a human voice frame, and adjacent human voice frames are merged to form a human voice segment.

[0021] In a preferred embodiment, in step S2, the fundamental frequency data of each vocal segment is calculated using an autocorrelation analysis method:

[0022] The autocorrelation function is calculated for the digitized discrete signal of a human voice segment to obtain the function value and form the autocorrelation function curve;

[0023] Search for the main peak position in the autocorrelation function curve of each voice segment, that is, the local peak with the largest amplitude except for zero delay, record the time delay corresponding to the local peak and take the reciprocal to obtain the fundamental frequency value;

[0024] The absolute value of the fundamental frequency difference between adjacent human voice segments is calculated as an acoustic deviation feature.

[0025] In a preferred embodiment, in step S2, when the acoustic deviation characteristics of adjacent voice segments are greater than a preset deviation threshold, it is determined that the two voice segments belong to different command issuance targets.

[0026] When the acoustic deviation characteristics of adjacent voice segments are less than or equal to a preset deviation threshold, the two voice segments are considered to be generated by the same command target.

[0027] By comparing preset deviation thresholds and acoustic deviation characteristics, different command issuance targets are established;

[0028] The duration of the voice segment corresponding to the target of each instruction is detected. The duration of the voice segment refers to the length of time from the start frame to the end frame in a single voice segment.

[0029] By comparing the duration of the voice output with a preset minimum effective duration threshold, voice segments with a duration greater than the preset minimum effective duration threshold are selected from the voice segments corresponding to the target of each instruction as valid instructions.

[0030] In a preferred embodiment, in step S3, the issuance time of each valid instruction is obtained by the instruction time acquisition unit, and the issuance times are sorted and combined in chronological order to form an issuance time set.

[0031] In the set of transmission times, the overlapping time is obtained by subtracting adjacent transmission times.

[0032] If the overlap time exceeds the preset overlap time threshold, the parallel instruction execution mechanism will be triggered.

[0033] Conversely, the overlapping instruction sorting mechanism is triggered.

[0034] In a preferred embodiment, in step S3, in the overlapping instruction sorting mechanism, the operating status value of the cooking device corresponding to the valid instruction is retrieved by the device control unit, and the operating status value represents the current operating status of the cooking device;

[0035] When the operating status value is 0, it indicates that the cooking equipment is in an idle standby state;

[0036] When the operating status value is 1, it indicates that the cooking equipment is in a switchable operating state;

[0037] When the operating status value is 2, it indicates that the cooking equipment is in operation.

[0038] In a preferred embodiment, in step S4, the valid instruction is processed by framing to obtain multiple frame audio segments, and the maximum amplitude value is extracted from the speech signal corresponding to the frame audio segment.

[0039] The effective audio amplitude is obtained by averaging the maximum amplitude values ​​in each audio segment.

[0040] Obtain registration target information from historical databases, including user registration time and cumulative number of command executions;

[0041] Subtracting the user's registration time from the current time yields the user's registration duration. Standardizing the registration duration yields a registration duration coefficient. The product of the registration duration coefficient and the cumulative number of command executions is used as the user activity index.

[0042] In a preferred embodiment, in step S4, the user activity index and the audio amplitude are standardized to obtain the user activity coefficient and the audio amplitude coefficient.

[0043] The priority level of the target issuing the command is calculated using the hyperbolic tangent function based on the user activity coefficient and the audio amplitude coefficient. ,in, and As a preset adjustment factor, For the preset bias term, User activity coefficient For audio amplitude coefficient, Priority level of the target to which the instruction is issued;

[0044] Retrieve the set of sent times, divide the valid instructions into time groups, and divide the valid instructions whose overlap time is less than or equal to the preset overlap time threshold into overlap groups.

[0045] Conversely, valid instructions with an overlap time greater than a preset overlap time threshold are classified into non-overlapping groups;

[0046] For valid instructions in non-overlapping groups, they are included in the instruction execution sequence in chronological order.

[0047] In a preferred embodiment, in step S4, the valid instructions in each overlapping group are sorted within the group according to the ascending order of their operating status values. If the operating status values ​​of the valid instructions are the same, the valid instructions are further sorted and adjusted according to the descending order of their priority levels.

[0048] After adjusting the order of the overlapping groups, the valid instructions of the overlapping groups are sequentially included in the instruction execution sequence according to the sorting result and the issuance time order of each overlapping group.

[0049] Execute valid instructions one by one in the order of the instruction execution sequence.

[0050] The technical effects and advantages of this invention are as follows:

[0051] This invention improves the response speed and accuracy of multi-task voice command processing in the kitchen by monitoring the ambient audio stream and dividing it into audio frames after the kitchen voice control device is turned on, extracting frame energy and zero-crossing rate to identify human voice segments, acquiring the fundamental frequency of the human voice segments and calculating acoustic deviation characteristics, creating command issuance targets based on the acoustic deviation, detecting the sound duration of each target to obtain valid commands, calculating the overlap time by retrieving the issuance time of valid commands, and selecting between parallel execution or overlapping sorting mechanisms based on the overlap time. In the overlapping sorting mechanism, the operating status and audio amplitude of the corresponding cooking equipment are obtained, and command priority levels are set in combination with historical registration information. Valid commands are sorted to generate an execution sequence and executed one by one in order. This improves the response speed and accuracy of multi-task voice command processing in the kitchen and reduces the risk of command conflicts and equipment misoperation. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the implementation of the method for dynamic recognition and separation of voice commands in a multi-person voice scenario according to the present invention.

[0053] Figure 2 This is a schematic diagram illustrating the steps of the method for dynamic recognition and separation of voice commands based on multi-person voice scenarios according to the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] This invention, after the kitchen voice control device is turned on, monitors the ambient audio stream and divides it into audio frames. It extracts frame energy and zero-crossing rate to identify human voice segments, collects the fundamental frequency of the human voice segments and calculates acoustic deviation characteristics, creates command issuance targets based on the acoustic deviation, detects the sound duration of each target to obtain valid commands, calculates the overlap time by retrieving the issuance time of valid commands, and selects to trigger parallel execution or overlapping sorting mechanism based on the overlap time. In the overlapping sorting mechanism, it obtains the operating status and audio amplitude of the corresponding cooking equipment, and sets the command priority level in combination with historical registration information. It sorts valid commands to generate an execution sequence and executes them one by one in order, thereby improving the response speed and accuracy of multi-task voice command processing in the kitchen.

[0056] Example 1, such as Figures 1 to 2 As shown, the method for dynamic recognition and separation of voice commands in multi-person voice scenarios includes the following steps:

[0057] Step S1: After the kitchen voice control device is turned on, it listens to the surrounding environment to obtain the ambient audio stream, divides the ambient audio stream into different audio frames, obtains the frame energy and zero-crossing rate of the audio frames, and extracts the human voice segments in the ambient audio stream by combining the frame energy and zero-crossing rate.

[0058] Step S2: Collect the fundamental frequency data of the human voice segment and calculate the acoustic deviation characteristics. Create different command issuance targets based on the acoustic deviation characteristics. Detect the sound duration of the human voice segment corresponding to each command issuance target. Collect the valid commands of each command issuance target based on the sound duration.

[0059] Step S3: Retrieve the issuance time of each valid instruction, calculate the overlap time of the valid instructions based on the issuance time, and select to trigger the parallel instruction execution mechanism or the overlapping instruction sorting mechanism based on the overlap time. In the overlapping instruction sorting mechanism, retrieve the operating status value of the cooking equipment corresponding to the valid instruction.

[0060] Step S4: Detect the audio amplitude of valid commands, obtain the registration target information through the historical database, set the priority level of the command issuing target by combining the registration target information and audio amplitude, sort the valid commands according to the operation status value and generate the command execution sequence, and execute the valid commands one by one according to the command execution sequence.

[0061] The specific implementation is as follows:

[0062] In step S1, the kitchen voice control device enters the listening mode after being turned on, and collects the acoustic signals of the surrounding environment in real time to obtain the environmental audio stream.

[0063] Ambient audio streams are continuous analog sound signals that vary over time, including speech signals, ambient noise, and other non-speech sound source signals;

[0064] The analog sound signal is digitized, that is, the analog sound signal is sampled at a fixed sampling rate by the built-in audio sampling module to obtain a digitized discrete signal sequence.

[0065] Performing a Fast Fourier Transform on a digitized discrete signal sequence maps the time-domain signal in the digitized discrete signal sequence to the frequency domain, obtaining the amplitude and phase of the corresponding frequency components of the digitized discrete signal sequence and forming a spectrum;

[0066] Based on the frequency index, the frequency components in the spectrum are sorted and partitioned, and the spectrum is divided into several continuous frequency sub-intervals, each containing multiple adjacent frequency components.

[0067] For each frequency sub-interval, read the amplitude spectrum value corresponding to each frequency component, and calculate the energy value of each frequency component by squaring. Sum the energy values ​​of all frequency components in the same frequency sub-interval and take the average to obtain the average energy value of the frequency sub-interval.

[0068] Based on the average energy value of each frequency sub-interval, and according to the preset noise frequency band threshold rules, the high-frequency components (such as wind noise and mechanical noise) and low-frequency components (such as power supply hum and environmental vibration noise) belonging to the noise band in the spectrum are initially filtered out to reduce the proportion of irrelevant energy interference in the spectrum.

[0069] It should be noted that the audio sampling module is a functional unit in the kitchen voice control equipment used to convert continuous analog sound signals into discrete digital signals suitable for digital signal processing. Its main function is to sample the input analog acoustic signal at regular time intervals using an analog-to-digital converter and quantize it into discrete amplitude data, thereby forming a digitized discrete signal sequence. The fixed sampling rate refers to the fixed sampling frequency used by the audio sampling module when digitizing analog signals, expressed in Hertz (Hz). The sampling rate setting satisfies the Nyquist sampling theorem, meaning the sampling rate should be at least twice the highest frequency component of the analog signal to avoid aliasing. In practical applications, for ambient audio streams containing speech signals, the fixed sampling rate can be set to 8000Hz, 16000Hz, or higher to ensure that the main frequency band of the speech signal (approximately 300Hz to 3400Hz) can be completely acquired. Fast Fourier Transmission (FFT)... The Lie transform is an algorithm that converts discrete-time domain signals into frequency domain representations. Its function is to map the amplitude information of audio signals that changes over time into the amplitude and phase information of each frequency component. The preset noise band threshold rule is the standard basis for identifying and filtering out non-speech signals in the spectrum. By setting high-frequency and low-frequency thresholds, the spectrum is divided into frequency bands that may contain noise and the main speech bands. The setting of high-frequency and low-frequency thresholds is determined based on the experimental measurement of environmental noise and the energy concentration range of speech signals, as well as the statistical results of typical noise characteristics in historical databases. By applying the noise band threshold rule, environmental noise is initially suppressed at the spectrum level, providing a clean speech signal basis for subsequent audio frame segmentation and human voice segment extraction.

[0070] The processed ambient audio stream is divided into multiple audio frames of equal length in chronological order. For each audio frame, its frame energy and zero-crossing rate are calculated.

[0071] Frame energy represents the energy intensity of an audio frame in the time domain, and its calculation formula is as follows:

[0072] E i = ∑ n=0 L-1 [ x i (n) ] 2 ;

[0073] in, The frame energy of the audio frame. The length of the audio frame. Let be the amplitude of the i-th audio frame at the n-th sampling point, where n is the index of the sampling point in the digitized discrete signal sequence, and i is the index of the audio frame.

[0074] Frame energy reflects the power intensity of the sound in an audio frame, and its value is significantly higher than that of a background noise frame.

[0075] The zero-crossing rate characterizes the number of times a digitized discrete signal waveform crosses the zero amplitude point within an audio frame, reflecting the speed of waveform oscillation. Its calculation formula is:

[0076] ;

[0077] in, The zero-crossing rate of the audio frame; The normalization coefficient is used to standardize the number of zero crossings to the frame length range, making the results between different frames comparable. The 2 in the denominator 2L is to offset the double counting effect generated in the absolute value difference. For symbolic functions, it is defined as follows: It is used to determine the sign of the current sampling point, that is, whether the signal at this point has a positive amplitude, zero amplitude, or negative amplitude; This represents the absolute value of the sign difference between two adjacent sampling points. If the two points have different signs (i.e., the signal crosses zero), the difference is 2; if the signs are the same, the difference is 0. The sum of the sign differences reflects the total number of times the signal waveform in the audio crosses the zero amplitude point, i.e., the zero-crossing rate.

[0078] The frame energy and zero-crossing rate are standardized to obtain the frame energy factor and zero-crossing rate factor. Specifically, the frame energy is divided by a preset energy reference value, and the zero-crossing rate is divided by a preset zero-crossing rate reference value.

[0079] The preset energy reference value is used to standardize the frame energy of each audio frame. It is determined based on the average frame energy of typical speech signals measured experimentally, reflecting the average power level of normal speech production, while also taking into account the intensity characteristics of the background noise environment. In specific applications, by collecting speech samples from multiple environments, the average value of each frame energy is obtained through statistical analysis and used as the energy reference value. This ensures that the standardized frame energy factor can characterize speech features without being biased by changes in environmental noise. The preset zero-crossing rate reference value is used to standardize the zero-crossing rate of each audio frame. It is set according to the statistical results of the zero-crossing rate of typical speech signals at the standard sampling rate, reflecting the high-frequency oscillation characteristics of normal speech. The specific setting method is to analyze the zero-crossing rate distribution of a large number of speech frames in static and typical noise environments, and select the median of the zero-crossing rate distribution as the zero-crossing rate reference value. This ensures that the standardized zero-crossing rate factor can characterize speech features and participate in human voice determination under different environmental conditions.

[0080] The voice evaluation value is obtained by adding the frame energy factor and the zero-crossing rate factor, and then compared with the preset voice threshold.

[0081] When the human voice evaluation value is greater than or equal to the preset human voice threshold, the audio frame is determined to be a human voice frame.

[0082] If the human voice evaluation value is less than the preset human voice threshold, the audio frame is determined to be a non-human voice frame.

[0083] Adjacent human voice frames are continuously detected and merged. Audio frames that continuously meet the judgment conditions are combined into human voice segments, thereby accurately extracting acoustic segments containing speech information from the environmental audio stream and providing stable speech signal input for subsequent speech recognition and command analysis.

[0084] In step S2, acoustic feature analysis is performed on the human voice segments to identify different command issuance targets and filter out valid commands.

[0085] First, the fundamental frequency is extracted from each voice segment. The fundamental frequency refers to the basic frequency of the vocal cord vibration during the voice production process, which reflects the physiological characteristics of the voice source and the periodic components of the speech signal.

[0086] The fundamental frequency data of each human voice segment is calculated by autocorrelation analysis. The autocorrelation function is calculated for the digitized discrete signal of each human voice segment. The autocorrelation function is a measure of the similarity between the digitized discrete signal and itself at different time delays. That is, the function value is obtained by accumulating the product of the digitized discrete signal and itself at each delay value. The function value reflects the periodic characteristics of the digitized discrete signal.

[0087] The function values ​​of a digitized discrete signal sequence form an autocorrelation function curve, where the location of the peak corresponds to the period of the repetition pattern of the digitized discrete signal.

[0088] Search for the main peak position in the autocorrelation function curve of each vocal segment, that is, the local peak with the largest amplitude except for zero delay. Record the time delay corresponding to the local peak. This delay value represents the period of vocal cord vibration. The fundamental frequency value can be obtained by taking its reciprocal. Use the fundamental frequency values ​​of all vocal segments as the fundamental frequency data.

[0089] It should be noted that autocorrelation analysis is an analytical method used to measure the similarity of a signal at different time delays. By calculating the self-multiplication and accumulation of the signal at each delay value, it can reveal the periodic characteristics and repetition patterns of the signal. In the field of speech processing, it is often used for fundamental frequency extraction. The periodic vibration of the speech signal is represented by periodic peaks in the autocorrelation function. By detecting the position of the main peak, the fundamental frequency can be obtained, thereby reflecting the basic frequency characteristics of the vocal cord vibration of the target when the command is issued.

[0090] The absolute value of the fundamental frequency difference between adjacent human voice segments is calculated and used as an acoustic deviation feature to quantify the differences in sound source characteristics between different human voice segments.

[0091] The comparison is performed based on the preset deviation threshold and acoustic deviation characteristics:

[0092] When the acoustic deviation characteristics of adjacent voice segments are greater than the preset deviation threshold, the two voice segments are determined to belong to different command issuance targets, which means that there are different individual sound sources.

[0093] When the acoustic deviation characteristics of adjacent voice segments are less than or equal to a preset deviation threshold, the two voice segments are considered to have been generated by the same command target.

[0094] The above comparison process establishes different command issuance targets, and all voice clips are categorized, with each category corresponding to an independent command issuance target.

[0095] The duration of the speech segment corresponding to each command target is detected. The speech duration refers to the time length from the start frame to the end frame in a single speech segment, reflecting the actual speech duration of the command target in the current speech input.

[0096] The duration of the speech is compared with the preset minimum effective duration threshold:

[0097] When the duration of the voice output exceeds the preset minimum effective duration threshold, the voice segment is determined to contain a valid instruction.

[0098] If the duration of the speech is less than or equal to the preset minimum effective duration threshold, the speech segment is determined to have no valid instruction, is defined as an invalid speech segment, and is discarded.

[0099] Based on the above judgment results, voice segments whose voice duration meets the judgment requirements are selected from the voice segments corresponding to the target of each instruction as valid instructions.

[0100] It should be noted that the preset deviation threshold is the fundamental frequency difference limit used to determine whether adjacent voice segments belong to the same command issuing target. It is determined based on experimental measurements of the fundamental frequency distribution range and variability of different command issuing targets under the same sampling rate and environmental conditions. By statistically analyzing the fundamental frequency difference data of multiple sets of typical voice segments, the difference value that can distinguish different voice individuals without misjudging the same voice individual is selected as the deviation threshold. The preset minimum effective duration threshold is the time length limit used to determine whether a voice segment constitutes a valid command. It is set based on the statistical data of the duration of typical command voices and the voice characteristics in the kitchen voice control environment. The minimum duration is determined as the minimum effective duration threshold by experimentally measuring the duration distribution of standard voice commands issued by multiple users.

[0101] In step S3, the issuance time of each valid instruction is obtained through the instruction time acquisition unit, and the issuance times are sorted and combined in chronological order to form an issuance time set. In the issuance time set, the difference between adjacent issuance times is processed to obtain the overlapping time.

[0102] Compare the overlap time with a preset overlap time threshold, and select to trigger either the parallel instruction execution mechanism or the overlap instruction sorting mechanism:

[0103] If the overlap time exceeds the preset overlap time threshold, the parallel instruction execution mechanism will be triggered.

[0104] Conversely, the overlapping instruction sorting mechanism is triggered.

[0105] When the overlap time is greater than the preset overlap time threshold, it is considered that the valid instructions are independent of each other, triggering the parallel instruction execution mechanism to issue and execute each valid instruction in chronological order; when the overlap time is less than or equal to the preset overlap time threshold, it indicates that the issuance time of adjacent valid instructions is relatively close, triggering the overlap instruction sorting mechanism for further analysis of the valid instructions.

[0106] In the overlapping instruction sorting mechanism, the operating status value of the cooking equipment corresponding to the valid instruction is retrieved by the device control unit. The operating status value indicates the current operating status of the cooking equipment. When the operating status value is 0, it means that the cooking equipment is in an idle standby state. When the operating status value is 1, it means that the cooking equipment is in a switchable operating state. When the operating status value is 2, it means that the cooking equipment is in a working operating state.

[0107] When the operating status value of the cooking equipment is 0, it means that the cooking equipment can immediately receive and execute new valid instructions; when the operating status value of the cooking equipment is 1, it means that the current task of the cooking equipment can be interrupted and new valid instructions can be executed; when the operating status value of the cooking equipment is 2, it means that the cooking equipment is working and cannot receive new control instructions.

[0108] For example, when the operating status value is 1, the criteria for determining this status include, but are not limited to: the task currently being performed by the device has a pause point (such as the heat preservation stage of an induction cooker or the preheating waiting stage of an oven), or the device supports a multi-task queue and the current queue is not full; in this state, the device can receive new valid instructions and can choose to execute them immediately or add them to the queue to be executed.

[0109] It should be explained that the instruction time acquisition unit is a component used to extract the time information of valid instructions; the preset overlap time threshold can be set according to the user's average speech rate or the speech interval of family members; the device control unit is a unit module for data interaction between various cooking devices, used to retrieve the operating status values ​​of the cooking devices.

[0110] This step extracts the issuance time of each valid command and determines the independence or overlap between commands based on the time difference between adjacent commands. It then retrieves the operating status value of the cooking equipment through the device control unit and determines whether the equipment can receive new commands based on the operating status value. This achieves intelligent judgment and scheduling of command conflicts in a multi-person voice environment, ensuring the orderliness of the control process and the safety of cooking.

[0111] In step S4, the valid command is processed by frame segmentation to obtain multiple frame audio segments. The maximum amplitude is extracted from the speech signal corresponding to the frame audio segment, and the average of the maximum amplitude in each frame audio segment is taken to obtain the audio amplitude of the valid command.

[0112] It should be explained that frame segmentation processing refers to dividing a continuous speech signal into several audio segments. In this embodiment, the valid instructions can be segmented according to a preset time length to extract the maximum amplitude value of the audio segments.

[0113] Audio amplitude reflects the strength of the voice signal. The larger the audio amplitude, the closer the target of the command is to the cooking equipment, thus increasing the priority of the command target; the smaller the audio amplitude, the farther the target of the command is from the equipment, thus decreasing the priority of the command target.

[0114] The registration target information is obtained through the historical database. The registration target information refers to the user identity and behavioral characteristics record data established when the user registers, including the user registration time and the cumulative number of commands executed.

[0115] Among them, the cumulative number of command executions refers to the total number of valid commands executed by the cooking equipment during the user's historical usage after registration, reflecting the user's activity and credibility. The earlier the user registered, the more fully they have learned and accumulated their voice characteristics and command habits, resulting in higher command stability and priority recognition and response in multi-person voice interaction scenarios.

[0116] Subtracting the user's registration time from the current time gives the user's registration duration. Standardizing the user's registration duration gives the registration duration coefficient. The earlier the user's registration time, the larger the registration duration coefficient.

[0117] The product of the registration duration coefficient and the cumulative number of command executions is used as the user activity index;

[0118] After standardizing the user activity index and audio amplitude, we obtain the user activity coefficient and audio amplitude coefficient, respectively. Using the user activity coefficient and audio amplitude coefficient, we calculate the priority level of the target issuing the command through the hyperbolic tangent function. ,in, and As a preset adjustment factor, For the preset bias term, User activity coefficient For audio amplitude coefficient, Priority level of the target to which the instruction is issued;

[0119] It should be noted that the hyperbolic tangent function has natural range compression and nonlinear suppression characteristics in priority calculation. On the one hand, the hyperbolic tangent function can limit the output range of the comprehensive score S to the (−1,1) interval, so that the scoring results from different feature sources have a unified dimension, which is convenient for normalization comparison in subsequent priority ranking and threshold determination. On the other hand, the hyperbolic tangent function tends to saturate when the absolute value of the input is large, and its output increase gradually slows down. It can effectively suppress the single dominant effect of any feature in user activity or audio amplitude on priority when it is abnormally high, thereby avoiding the priority being monopolized by a certain high-value feature and improving the stability and balance of the overall judgment process.

[0120] Among them, the regulating factor and This is used to balance the impact of user activity and voice amplitude on priority. For example, in scenarios that emphasize user identity, a weighting can be set. In scenarios where voice clarity and proximity are important, settings can be configured. The bias term b is used to adjust the baseline level of priority. The specific values ​​of these parameters can be determined by regression analysis or grid search optimization using multi-person voice command data collected in a typical kitchen environment.

[0121] Retrieve the set of sent times, divide the valid instructions into time groups, and divide the valid instructions whose overlap time is less than or equal to the preset overlap time threshold into overlapping groups, and conversely, divide the valid instructions whose overlap time is greater than the preset overlap time threshold into non-overlapping groups.

[0122] For valid instructions in non-overlapping groups, they are included in the instruction execution sequence in chronological order.

[0123] For each overlapping group, valid instructions are sorted based on priority and operational status value before being included in the instruction execution sequence:

[0124] Valid instructions are sorted within a group according to their operating status values ​​in ascending order. If valid instructions have the same operating status value, they are further sorted and adjusted according to their priority in descending order.

[0125] After sorting within the group, the effective instructions of the overlapping groups are sequentially incorporated into the instruction execution sequence according to the sorting results and the issuance time order of each group.

[0126] Valid instructions are executed one by one in the order of the instruction execution sequence to achieve time-series fusion and priority scheduling of multi-user voice instructions.

[0127] It should be noted that the historical database is a long-term data storage module used to store user registration information and instruction execution records. In this embodiment, it is used to obtain registration target information. The standardization processing method includes, but is not limited to, standard linear transformation based on interval scaling, Z-Score standardization method based on statistics, or normalization method based on nonlinear mapping function. The application method of standardization processing will not be elaborated here. The preset adjustment factor can be set according to the usage environment and user characteristics. For example, when focusing on voice distance, the adjustment factor corresponding to the audio amplitude coefficient can be increased. The preset bias term is used to adjust the center position of the output and can be set according to historical running data or user distribution.

[0128] This step obtains the audio amplitude by framing the voice signal of valid commands, calculates the user activity index by combining it with the registered target information, and generates the priority level of the command issuing target. The command priority is distinguished based on multi-dimensional features such as user registration information, usage frequency, and voice distance. By combining the priority level with the device operation status value for sorting, the execution order can be accurately determined when multiple people issue commands at the same time, reducing command conflicts and false triggers, and realizing intelligent response and safety control of cooking equipment.

[0129] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0130] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0131] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0132] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0133] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for dynamic recognition and separation of voice commands in multi-person voice scenarios, characterized in that: Includes the following steps: Step S1: After the kitchen voice control device is turned on, it listens to the surrounding environment to obtain the ambient audio stream, divides the ambient audio stream into different audio frames, obtains the frame energy and zero-crossing rate of the audio frames, and extracts at least one human voice segment from the ambient audio stream by combining the frame energy and zero-crossing rate. Step S2: Collect the fundamental frequency data of the human voice segment and calculate the acoustic deviation characteristics. Create different command issuance targets based on the acoustic deviation characteristics. Detect the sound duration of the human voice segment corresponding to each command issuance target. Collect the valid commands of each command issuance target based on the sound duration. In step S2, the fundamental frequency data of each vocal segment is calculated using autocorrelation analysis: The autocorrelation function is calculated for the digitized discrete signal of a human voice segment to obtain the function value and form the autocorrelation function curve; Search for the main peak position in the autocorrelation function curve of each voice segment, that is, the local peak with the largest amplitude except for zero delay, record the time delay corresponding to the local peak and take the reciprocal to obtain the fundamental frequency value; The absolute value of the fundamental frequency difference between adjacent human voice segments is calculated as an acoustic deviation feature. In step S2, when the acoustic deviation characteristics of adjacent voice segments are greater than a preset deviation threshold, it is determined that the two voice segments belong to different command issuance targets. When the acoustic deviation characteristics of adjacent voice segments are less than or equal to a preset deviation threshold, the two voice segments are considered to be generated by the same command target. By comparing preset deviation thresholds and acoustic deviation characteristics, different command issuance targets are established; The duration of the voice segment corresponding to the target of each instruction is detected. The duration of the voice segment refers to the length of time from the start frame to the end frame in a single voice segment. By comparing the duration of the voice output with a preset minimum effective duration threshold, voice segments with a duration greater than the preset minimum effective duration threshold are selected from the voice segments corresponding to the target of each instruction as valid instructions. Step S3: Retrieve the issuance time of each valid instruction, calculate the overlap time of the valid instructions based on the issuance time, and select to trigger the parallel instruction execution mechanism or the overlapping instruction sorting mechanism based on the overlap time. In the overlapping instruction sorting mechanism, retrieve the operating status value of the cooking equipment corresponding to the valid instruction. Step S4: Detect the audio amplitude of valid commands, obtain the registration target information through the historical database, set the priority level of the command issuing target by combining the registration target information and audio amplitude, sort the valid commands by combining the operation status value and generate the command execution sequence, and execute the valid commands one by one according to the command execution sequence.

2. The method for dynamic recognition and separation of voice commands based on multi-person voice scenarios according to claim 1, characterized in that: In step S1, the kitchen voice control device enters the listening mode after being turned on, and collects the acoustic signals of the surrounding environment in real time to obtain the environmental audio stream. Ambient audio streams are continuous analog sound signals that vary over time, including speech signals, ambient noise, and other non-speech sound source signals; The analog sound signal is digitized, that is, the analog sound signal is sampled at a fixed sampling rate by the built-in audio sampling module to obtain a digitized discrete signal sequence. A fast Fourier transform is performed on the digitized discrete signal sequence, and high-frequency and low-frequency components belonging to the noise region in the spectrum are initially filtered out according to the preset noise frequency band threshold rules.

3. The method for dynamic recognition and separation of voice commands based on multi-person voice scenarios according to claim 1, characterized in that: The processed ambient audio stream is divided into multiple audio frames of equal length in chronological order. For each audio frame, the frame energy and zero-crossing rate are calculated. Frame energy characterizes the energy intensity of an audio frame in the time domain; The zero-crossing rate characterizes the number of times the waveform of a digitized discrete signal crosses the zero amplitude point within an audio frame; The frame energy and zero-crossing rate are standardized separately to obtain the frame energy factor and zero-crossing rate factor. The frame energy factor and zero-crossing rate factor are added together to obtain the human voice evaluation value. When the human voice evaluation value is greater than or equal to the preset human voice threshold, the audio frame is determined to be a human voice frame, and adjacent human voice frames are merged to form a human voice segment.

4. The method for dynamic recognition and separation of voice commands based on multi-person voice scenarios according to claim 1, characterized in that: In step S3, the issuance time of each valid instruction is obtained through the instruction time acquisition unit, and the issuance times are sorted and combined in chronological order to form an issuance time set. In the set of transmission times, the overlapping time is obtained by subtracting adjacent transmission times. If the overlap time exceeds the preset overlap time threshold, the parallel instruction execution mechanism will be triggered. Conversely, the overlapping instruction sorting mechanism is triggered.

5. The method for dynamic recognition and separation of voice commands based on multi-person voice scenarios according to claim 1, characterized in that: In step S3, in the overlapping instruction sorting mechanism, the operating status value of the cooking equipment corresponding to the valid instruction is retrieved by the device control unit, and the valid instructions are initially sorted. The operating status value represents the current operating status of the cooking equipment. When the operating status value is 0, it indicates that the cooking equipment is in an idle standby state; When the operating status value is 1, it indicates that the cooking equipment is in a switchable operating state; When the operating status value is 2, it indicates that the cooking equipment is in operation.

6. The method for dynamic recognition and separation of voice commands based on multi-person voice scenarios according to claim 1, characterized in that: In step S4, the valid instruction is processed by frame segmentation to obtain multiple frame audio segments, and the maximum amplitude value is extracted from the speech signal corresponding to the frame audio segment. The effective audio amplitude is obtained by averaging the maximum amplitude values ​​in each audio segment. Obtain registration target information from historical databases, including at least user registration time and cumulative command execution count; Subtracting the user's registration time from the current time yields the user's registration duration. Standardizing the registration duration yields a registration duration coefficient. The product of the registration duration coefficient and the cumulative number of command executions is used as the user activity index.

7. The method for dynamic recognition and separation of voice commands based on multi-person voice scenarios according to claim 6, characterized in that: In step S4, the user activity index and audio amplitude are standardized to obtain the user activity coefficient and audio amplitude coefficient, respectively. The priority level of the target issuing the command is calculated using the hyperbolic tangent function based on the user activity coefficient and the audio amplitude coefficient. ,in, and As a preset adjustment factor, For the preset bias term, User activity coefficient For audio amplitude coefficient, Priority level of the target to which the instruction is issued; Retrieve the set of sent times, divide the valid instructions into time groups, and divide the valid instructions whose overlap time is less than or equal to the preset overlap time threshold into overlap groups. Conversely, valid instructions with an overlap time greater than a preset overlap time threshold are classified into non-overlapping groups; For valid instructions in non-overlapping groups, they are included in the instruction execution sequence in chronological order.

8. The method for dynamic recognition and separation of voice commands based on multi-person voice scenarios according to claim 7, characterized in that: In step S4, for each overlapping group of valid instructions, the valid instructions are sorted within the group according to the ascending order of their operating status values. If the operating status values ​​of the valid instructions are the same, the valid instructions are further sorted and adjusted according to the descending order of their priority levels. After adjusting the order of the overlapping groups, the valid instructions of the overlapping groups are sequentially included in the instruction execution sequence according to the sorting result and the issuance time order of each overlapping group. Execute valid instructions one by one in the order of the instruction execution sequence.

Images