Voice control method and device, intelligent earphone and voice control system

Abstract

The application relates to the field of intelligent earphones, and discloses a voice control method and device, an intelligent earphone and a voice control system. The method comprises the following steps: in the case of receiving a voice instruction, determining whether the voice instruction meets local processing requirements, wherein the local processing requirements are requirements supported by a first recognition model; if yes, recognizing the voice instruction through the first recognition model, generating a first visual interaction instruction, and executing an operation corresponding to the first visual interaction instruction; if not, uploading the voice instruction to a cloud device, receiving a second visual interaction instruction sent by the cloud, and executing an operation corresponding to the second visual interaction instruction, wherein the second visual interaction instruction is generated by recognizing the voice instruction through a second recognition model. The application can reduce time consumption and improve user experience.

Description

Technical Field

[0001] This application relates to the field of smart headphones, and more particularly to a voice control method, device, smart headphones, and voice control system. Background Technology

[0002] In the field of wireless audio devices, especially true wireless stereo headphones and audio modules for augmented reality / virtual reality devices, integrating intelligent voice control has become a key technology for improving user experience. Currently, mainstream implementations generally adopt an "edge-cloud collaborative" architecture: the device (such as headphones) collects the user's voice signal (e.g., "take a picture" or "identify what's in front?"), and uploads the raw audio data to a remote cloud server. The cloud deploys a large-scale, high-precision automatic speech recognition model responsible for transcribing the speech to text and parsing the semantics of the commands. Finally, it sends the recognized visual interaction commands (e.g., commands to take a picture or to announce the recognition results) back to the device for execution (e.g., the headphones take a picture after receiving the command).

[0003] However, existing voice commands, even the simplest ones like "take a photo" or "pause," must go through a complete network loop of "audio upload - cloud processing - command delivery," which takes a long time and seriously affects the interactive experience that requires a fast response (such as taking photos), resulting in a poor user experience. Summary of the Invention

[0004] The main purpose of this application is to provide a voice control method that aims to solve the technical problem of how to improve the user experience during voice control.

[0005] To achieve the above objectives, this application proposes a voice control method, which is applied to a smart earphone in a voice control system, wherein a first recognition model is deployed in the smart earphone; The voice control system further includes: a cloud device, in which a second recognition model is deployed, wherein the model size of the first recognition model is smaller than that of the second recognition model, and the smart earphone is connected to the cloud device; The method includes: Upon receiving a voice command, it is determined whether the voice command meets the local processing requirements, which are the requirements that the first recognition model supports for recognition. If the conditions are met, the voice command is recognized by the first recognition model, a first visual interaction command is generated, and the operation corresponding to the first visual interaction command is executed. If the conditions are not met, the voice command is uploaded to the cloud device, the second visual interaction command sent by the cloud is received, and the operation corresponding to the second visual interaction command is executed. The second visual interaction command is generated by recognizing the voice command through the second recognition model.

[0006] In one embodiment, the step of determining whether the voice command meets the local processing requirements includes: The voice command is muted, the duration of the muted segment is determined, and it is determined whether the duration is not higher than a preset duration threshold. And / or, determine the text length of the voice command, estimate the number of words based on the text length, and determine whether the number of words is not higher than a preset number threshold; And / or, determine the current keyword in the voice command, and determine whether the current keyword is a preset keyword.

[0007] In one embodiment, the smart earphones further include a vibration sensing component; The step of determining whether a received voice command meets local processing requirements includes: Upon receiving a voice command, the vibration sensing component detects vibrations from the wearer and obtains vibration signals. If the vibration signal meets the preset sound emission requirements, determine whether the voice command meets the local processing requirements.

[0008] In one embodiment, the step of determining whether the voice command meets local processing requirements when the vibration signal meets preset sound emission requirements includes: The vibration signal and the voice command are time-domain aligned, and the aligned vibration signal is feature-extracted to obtain the current vibration waveform, current vibration frequency, and current vibration intensity. Based on the current vibration waveform, the current vibration frequency, the current vibration intensity, and the aligned voice command, determine whether the vibration signal meets the preset voice-accompanied vibration requirements; If so, determine whether the voice command meets the local processing requirements.

[0009] In one embodiment, the step of recognizing the voice command through the first recognition model and generating a first visual interaction command includes: A reference noise spectrum is generated based on the aligned vibration signal; The speech command is adaptively filtered using the reference noise spectrum. The first recognition model recognizes the filtered voice commands and generates the first visual interaction command.

[0010] In one embodiment, the step of generating a reference noise spectrum based on the aligned vibration signal includes: The aligned vibration signal is framed, and then windowed. Determine the short-time energy of each frame in the windowed vibration signal, select the short-time energy corresponding to a preset number of frames, and determine the energy threshold based on the short-time energy corresponding to the preset number of frames. The energy threshold is compared with the short-time energy of each frame to obtain the voice activity frame; Perform a Fourier transform on each of the aforementioned speech activity frames, and calculate the power spectral density based on each transformed speech activity frame to obtain the vibration power spectrum. A reference noise spectrum is generated based on the vibration power spectrum of each vibration.

[0011] In one embodiment, the step of generating a reference noise spectrum based on each of the vibration power spectra includes: Each vibration power spectrum is weighted by a preset frequency band weighting mapping function; The time-averaged spectrum of each weighted vibration power spectrum is then smoothed in the frequency domain. The power spectrum of each vibration after frequency domain smoothing is normalized to obtain the reference noise spectrum.

[0012] Furthermore, to achieve the above objectives, this application also proposes a voice control device, the device comprising: The judgment module is used to determine whether the voice command meets the local processing requirements when a voice command is received, wherein the local processing requirements are the requirements that the first recognition model supports for recognition. The first recognition module is configured to, if satisfied, recognize the voice command through the first recognition model, generate a first visual interaction command, and execute the operation corresponding to the first visual interaction command. The second recognition module is used to upload the voice command to the cloud device if the conditions are not met, receive the second visual interaction command sent by the cloud, and execute the operation corresponding to the second visual interaction command. The second visual interaction command is generated by recognizing the voice command through the second recognition model.

[0013] In addition, to achieve the above objectives, this application also proposes a smart earphone, wherein a first recognition model is deployed within the smart earphone; The smart earphone further includes: a memory, a processor, and a voice control program stored in the memory and executable on the processor, wherein the voice control program, when executed by the processor, implements the steps of the voice control method as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a voice control system, which includes: a cloud device and a smart headset as described above; The cloud device has a second recognition model deployed within it. The model size of the first recognition model is smaller than that of the second recognition model. The smart earphones are connected to the cloud device.

[0015] This application proposes a voice control method, device, smart earphone, and voice control system. The method is applied to a smart earphone in a voice control system, wherein a first recognition model is deployed within the smart earphone. The voice control system further includes a cloud device, wherein a second recognition model is deployed within the cloud device, the model size of the first recognition model being smaller than that of the second recognition model, and the smart earphone being connected to the cloud device. The method includes: upon receiving a voice command, determining whether the voice command meets local processing requirements, wherein the local processing requirements are the requirements supported by the first recognition model; if the requirements are met, then recognizing the voice command through the first recognition model to generate a first visual interaction command, and executing the operation corresponding to the first visual interaction command; if the requirements are not met, then uploading the voice command to the cloud device, receiving a second visual interaction command sent from the cloud, and executing the operation corresponding to the second visual interaction command, wherein the second visual interaction command is generated by recognizing the voice command through the second recognition model.

[0016] This application incorporates a local first recognition model within the smart earphone and a second recognition model on a cloud-based device. Upon receiving a voice command, it first determines whether the command meets the local processing requirements. During actual control, if the voice command matches the capabilities of the local model, it directly recognizes and executes the corresponding operation; if it exceeds the local processing range, it is uploaded to the cloud for processing by the more powerful second recognition model, which then returns a visual interaction command. Compared to existing methods that require uploading to a cloud device for recognition, this application utilizes the dynamic collaboration between the local first recognition model and the cloud-based second recognition model. The first recognition model ensures rapid response to basic commands, while the second recognition model handles complex commands, thereby reducing processing time and improving the user experience. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a block diagram of the voice control system structure proposed in the first embodiment of this application; Figure 2 This is a flowchart of the first embodiment of the voice control method proposed in this application; Figure 3 This is a flowchart illustrating the judgment process in the voice control method proposed in this application. Figure 4 This is a flowchart of the second embodiment of the voice control method proposed in this application; Figure 5 This is a diagram illustrating the overall framework of the voice control method proposed in this application. Figure 6 This is a flowchart of the third embodiment of the voice control method proposed in this application; Figure 7 A diagram of a voice control device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a smart earphone suitable for implementing the embodiments of this application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0024] Understandably, in the field of wireless audio devices, especially true wireless stereo headphones and audio modules for augmented reality / virtual reality devices, integrating intelligent voice control has become a key technological direction for improving user experience. Currently, the mainstream implementation generally adopts an "edge-cloud collaborative" architecture: the device (such as headphones) collects the user's voice signal (e.g., "take a picture" or "identify what's in front?"), and uploads the raw audio data to a remote cloud server. The cloud deploys a large-scale, high-precision automatic speech recognition model responsible for transcribing speech to text and parsing the semantics of the commands. Finally, it sends the recognized visual interaction commands (e.g., commands to take a picture or to announce the recognition results) back to the device for execution (e.g., the headphones take a picture after receiving the command).

[0025] However, existing voice commands, even the simplest ones like "take a photo" or "pause," must go through a complete network loop of "audio upload - cloud processing - command delivery," which takes a long time and seriously affects the interactive experience that requires a fast response (such as taking photos), resulting in a poor user experience.

[0026] Therefore, to address the technical challenge of improving user experience during voice control, this embodiment proposes a voice control method. This method is applied to a smart headset in a voice control system, where a first recognition model is deployed. The voice control system also includes a cloud device, where a second recognition model is deployed. The first recognition model has a smaller model size than the second recognition model. The smart headset is connected to the cloud device. The method includes: upon receiving a voice command, determining whether the voice command meets local processing requirements, which are the requirements supported by the first recognition model; if met, recognizing the voice command through the first recognition model to generate a first visual interaction command and executing the operation corresponding to the first visual interaction command; if not met, uploading the voice command to the cloud device, receiving a second visual interaction command sent from the cloud, and executing the operation corresponding to the second visual interaction command, whereby the second visual interaction command is generated by recognizing the voice command through the second recognition model.

[0027] This embodiment incorporates a local first recognition model within the smart earphone and a second recognition model on a cloud device. Upon receiving a voice command, it first determines whether the command meets the local processing requirements. During actual control, if the voice command matches the capabilities of the local model, it directly recognizes and executes the corresponding operation; if it exceeds the local processing range, it is uploaded to the cloud for processing by the more powerful second recognition model, which then returns a visual interaction command. Compared to existing methods that require uploading to the cloud device for recognition, this embodiment utilizes the dynamic collaboration between the local first recognition model and the cloud-based second recognition model. The first recognition model ensures rapid response to basic commands, while the second recognition model handles complex commands, thereby reducing processing time and improving the user experience.

[0028] For ease of understanding, the following is combined with Figures 1 to 8 The voice control method provided in the embodiments of this application, as well as the voice control method, device, smart earphone, and voice control system provided in the following embodiments, will be described in detail.

[0029] This application provides a voice control method, which is applied to a smart headset. (See reference...) Figure 1 , Figure 1 This is a block diagram of the voice control system structure proposed in the first embodiment of this application. The above method is applied to a smart headset in the voice control system, wherein a first recognition model is deployed within the smart headset. Furthermore, as... Figure 1 As shown, the voice control system further includes a cloud device, in which a second recognition model is deployed. The model size of the first recognition model is lower than that of the second recognition model. The smart earphone is connected to the cloud device. The smart earphone can connect directly to the cloud device, or through an earphone case, or through a mobile device (e.g., a mobile phone). This embodiment does not impose any limitations.

[0030] It is understood that the aforementioned smart earphones can be wearable electronic products integrating computing units, sensors, and wireless communication modules, such as true wireless stereo earphones. The aforementioned first recognition model can be a compressed and optimized lightweight artificial intelligence model deployed on the local processor of the smart earphones, with a small number of parameters, computational complexity, and storage footprint, specifically designed to recognize a limited set of preset simple voice commands. The aforementioned mobile device can be a smart terminal paired with the smart earphones via short-range wireless communication technology (such as Bluetooth), such as a smartphone, tablet, or laptop, primarily acting as a communication relay and data processing auxiliary node between the smart earphones and the wide area network. The aforementioned cloud device can be a remote server or server cluster deployed in an internet data center with powerful computing and storage resources. The second recognition model running on it is a larger-scale, more versatile heavy-duty artificial intelligence model capable of handling complex and open-ended speech recognition and natural language understanding tasks. Therefore, in this embodiment, the model size of the first recognition model is smaller than that of the second recognition model.

[0031] Reference Figure 2 , Figure 2 This is a flowchart of the first embodiment of the voice control method proposed in this application.

[0032] like Figure 2 As shown, the method includes: Step S10: Upon receiving a voice command, determine whether the voice command meets the local processing requirements, wherein the local processing requirements are the requirements that the first recognition model supports for recognition.

[0033] It should be noted that the executing entity in this embodiment can be a multifunctional machine device with voice control (i.e., the aforementioned smart headset), such as a smart headset integrating a camera and microphone, or a device capable of realizing the above functions. This embodiment uses a smart headset with a first recognition model (hereinafter referred to as the device) for description, but does not impose specific limitations on this embodiment.

[0034] It is understandable that the aforementioned voice command can be a voice signal actively issued by the user to control the device, containing a specific intention to take pictures, such as a command to take pictures or recognize content in front. The aforementioned local processing requirements can be a set of pre-defined requirements used to determine whether a voice command is suitable and can be processed by the first recognition model deployed locally on the smart earphone. The aforementioned requirements for supporting recognition can be the capability range requirements defined by the first recognition model during the design and training phases, such as: the set of command keywords that can be accurately recognized, the grammatical structure of the command that can be understood (sentence length, complexity), and the acoustic conditions (such as signal-to-noise ratio) that can achieve a satisfactory recognition rate. In specific implementation, after acquiring a voice command, the device evaluates whether the features of the current voice command fall within the capability range of the first recognition model, that is, whether it meets the local processing requirements.

[0035] Step S20: If satisfied, the voice command is recognized by the first recognition model, a first visual interaction command is generated, and the operation corresponding to the first visual interaction command is executed.

[0036] It is understood that the aforementioned first visual interaction instruction can be a structured machine-readable instruction output by the first recognition model, which can be used to clarify the specific operation corresponding to the user's visual-related operation intention, such as taking a picture or broadcasting the recognition result of the object in front.

[0037] In its implementation, once the device determines that the voice command meets the local processing requirements, it immediately initiates the local recognition and execution process. The device inputs the voice data into its internally deployed first recognition model for inference. The model outputs a structured recognition result, i.e., the first visual interaction command. Subsequently, the device's control logic parses the command and directly drives its own hardware to complete the operation required by the command.

[0038] For ease of understanding, the following example is used for illustration, but it does not impose any specific limitations on this embodiment. When a user says the short command "take a picture" to the earphone, the earphone determines that it meets the local processing requirements, and immediately calls the built-in lightweight model to recognize and generate the first visual interaction command "trigger shooting". The earphone then takes a picture, and the whole process is completed instantly without going through the cloud.

[0039] Step S30: If not satisfied, the voice command is uploaded to the cloud device, the second visual interaction command sent by the cloud is received, and the operation corresponding to the second visual interaction command is executed. The second visual interaction command is generated by recognizing the voice command through the second recognition model.

[0040] It is understood that the aforementioned second visual interaction instruction can be a structured machine-readable instruction generated by the second recognition model, used to clarify the specific operation corresponding to the user's intent.

[0041] In its implementation, the aforementioned device initiates a cloud processing path after determining that the voice command does not meet local processing requirements. The smart earphones then wirelessly transmit the voice command data to the cloud device. The cloud device uses its deployed second recognition model to recognize the voice command, generates a second visual interaction command, and sends the command to the smart earphones, or directly controls itself or other associated devices to perform corresponding operations based on the command content.

[0042] For ease of understanding, the following example illustrates the concept, but does not limit the scope of this embodiment. When a user says "Identify what's ahead" to the earphones, because this command is complex and contains key information not pre-set, the earphones determine that it does not meet local processing requirements and upload it to the cloud device via the mobile network. The cloud-based second recognition model accurately identifies and semantically understands the command, generating a second visual interaction command containing information such as the specific category, state, and relative distance of the object in front, and sends it to the smart earphones. The smart earphones then play the command according to the second visual interaction command.

[0043] refer to Figure 3 , Figure 3 This is a flowchart illustrating the judgment process in the voice control method proposed in this application. (Reference) Figure 3 Furthermore, in order to determine whether the local processing requirements are met, the step of determining whether the voice command meets the local processing requirements includes: Step S11: Remove the silence segment from the voice command, determine the duration of the removed voice command, and determine whether the duration is not higher than a preset duration threshold.

[0044] It should be noted that the aforementioned duration refers to the total length of the remaining continuous speech activity on the time axis after removing all silent segments from the speech command signal, usually measured in milliseconds or seconds. The aforementioned preset duration threshold can be a pre-set time value used as an objective standard for judging whether a speech command is "brief." This threshold is set based on statistical analysis of a large number of simple commands and the processing capability of the first recognition model.

[0045] In its implementation, the device first uses an endpoint detection algorithm to locate and remove silent segments at the beginning, end, and middle of the audio. Then, it calculates the total duration of the effective speech segment after removing silence. Finally, it compares the calculated duration with a preset duration threshold stored internally (i.e., ...). Figure 3The process calculates the duration of the speech signal and determines whether the duration exceeds a preset threshold.

[0046] If the duration of the above-mentioned event exceeds the preset duration, it is determined that the voice command does not meet the local processing requirements, and the voice command is uploaded to the cloud device (i.e., Figure 3 (Cloud processing in the cloud), receiving the second visual interaction command sent by the cloud, and executing the operation corresponding to the second visual interaction command (i.e. Figure 3 The earphones receive the command and execute the corresponding action. If the duration of the above-mentioned event is no greater than a preset duration, it is determined that the voice command meets the local processing requirements, and the voice command is recognized by the first recognition model (i.e., ...). Figure 3 The earphones process the language locally and send it to the local semantic recognition model to generate the first visual interaction command (i.e., Figure 3 The model outputs instructions. And executes the operation corresponding to the first visual interaction instruction (i.e.,...). Figure 3 The earpiece receives the instruction and executes the corresponding action.

[0047] Step S12: and / or, determine the text length of the voice command, estimate the number of words based on the text length, and determine whether the number of words is not higher than a preset threshold.

[0048] Understandably, the aforementioned text length can refer to the number of characters, syllables, or phonemes contained in the text after the voice command is recognized or estimated and transcribed, serving as a primary indicator of command complexity. The aforementioned word count can refer to the number of independent semantic units (i.e., words or characters) contained in the text segment, based on text length, syllable rhythm, or estimated through a minimal language model. The aforementioned preset threshold can be a pre-defined numerical upper limit used to define the range of word counts belonging to "simple commands," for example, commands containing no more than 3 keywords are typically considered suitable for local processing.

[0049] In its implementation, the aforementioned device first estimates the text length corresponding to the instruction (i.e., syllable counting or lightweight speech feature analysis) through basic syllable counting or lightweight speech feature analysis. Figure 3 The text length is detected in the text, and then the number of words that the instruction may contain is further estimated based on the text length or a specific speech rhythm model (i.e., the number of words). Figure 3 The estimated number of words in the text is then compared with a preset threshold (i.e., the estimated number of words). Figure 3 (Whether the number of words exceeds a preset threshold).

[0050] If the number of words is not higher than a preset threshold, the voice command is determined to meet the local processing requirements. The voice command is then recognized using the first recognition model (i.e.,...). Figure 3The earphones process the language locally and send it to the local semantic recognition model to generate the first visual interaction command (i.e., Figure 3 The model outputs instructions. And executes the operation corresponding to the first visual interaction instruction (i.e.,...). Figure 3 The earphones receive the command and execute the corresponding action. If the number of words exceeds a preset threshold, the voice command is determined not to meet local processing requirements, and the voice command is uploaded to the cloud device (i.e., Figure 3 (Cloud processing in the cloud), receiving the second visual interaction command sent by the cloud, and executing the operation corresponding to the second visual interaction command (i.e. Figure 3 The earpiece receives the instruction and executes the corresponding action.

[0051] Step S13: and / or, determine the current keyword in the voice command, and determine whether the current keyword is a preset keyword.

[0052] It should be noted that the aforementioned current keywords may refer to one or more core words with a clear control intent extracted or identified from the voice command. The aforementioned preset keywords may be a limited set of words pre-stored locally on the device, which contains all the core words of the commands that the first recognition model can accurately identify and process, such as "play," "pause," "take a picture," "mute," "next song," etc.

[0053] In its implementation, the aforementioned device performs keyword matching analysis on voice commands: through rapid feature matching or a very small trigger word detection model, it extracts one or more core words from the voice command as current keywords. These current keywords are then compared with a locally stored set of preset keywords (i.e.,...). Figure 3 Keyword detection in the code, and determination of whether it is a preset command word.

[0054] If the current keyword is a preset keyword, the voice command is determined to meet local processing requirements. The aforementioned device recognizes the voice command using the first recognition model (i.e., Figure 3 The earphones process the language locally and send it to the local semantic recognition model to generate the first visual interaction command (i.e., Figure 3 The model outputs instructions. And executes the operation corresponding to the first visual interaction instruction (i.e.,...). Figure 3 The earphones receive the command and execute the corresponding action. If the current keyword is not a preset keyword, it is determined that the voice command does not meet the local processing requirements, and the voice command is uploaded to the cloud device (i.e., Figure 3 (Cloud processing in the cloud), receiving the second visual interaction command sent by the cloud, and executing the operation corresponding to the second visual interaction command (i.e. Figure 3 The earpiece receives the instruction and executes the corresponding action.

[0055] This embodiment incorporates a local first recognition model within the smart earphone and a second recognition model on a cloud device. Upon receiving a voice command, it first determines whether the command meets the local processing requirements. During actual control, if the voice command matches the capabilities of the local model, it directly recognizes and executes the corresponding operation; if it exceeds the local processing range, it is uploaded to the cloud for processing by the more powerful second recognition model, which then returns a visual interaction command. Compared to existing methods that require uploading to the cloud for recognition, this embodiment utilizes the dynamic collaboration between the local first recognition model and the cloud-based second recognition model. The first recognition model ensures rapid response to basic commands, while the second recognition model handles complex commands. Therefore, simple commands can be directly processed by the local first recognition model without uploading, reducing processing time and improving the user experience.

[0056] Based on the first embodiment, in the second embodiment, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart of the second embodiment of the voice control method proposed in this application. Furthermore, in order to determine whether the command was issued by the user, the smart earphone is also equipped with a vibration sensing component.

[0057] It is understood that the aforementioned vibration sensing component may be a sensor integrated into the smart earphone for detecting physical micro-vibration signals generated by human voice or activity, such as a highly sensitive accelerometer, a bone conduction sensor specifically designed to collect vibrations transmitted through bones, or an electromyography sensor capable of detecting minute vibrations on the skin surface.

[0058] refer to Figure 5 , Figure 5 This is a diagram illustrating the overall framework of the voice control method proposed in this application. Figure 5 As shown, the step of determining whether a voice command meets local processing requirements upon receiving a voice command includes: Step S101: Upon receiving a voice command, the vibration sensing component senses the vibration of the wearer and obtains a vibration signal; Step S102: If the vibration signal meets the preset sound requirements, determine whether the voice command meets the local processing requirements.

[0059] It should be noted that the aforementioned vibration signal can be physical signal waveform data collected by the vibration sensing component, reflecting specific physiological activities of the human body (such as vibration of the vocal cords causing vibration transmitted through bones or skin). The aforementioned preset vocalization requirements can be a set of pre-set characteristic conditions or thresholds used to determine whether the vibration signal is generated by human active vocalization. These conditions or thresholds are set based on the analysis of the waveform, energy, dominant frequency, and other characteristics of typical speech-related vibration signals.

[0060] refer to Figure 5 In its specific implementation, upon receiving a voice command, the aforementioned device simultaneously activates the vibration sensing component to collect vibration signals (i.e., Figure 5 (Synchronous acquisition of multimodal speech signals). The above-mentioned equipment analyzes the characteristics of the vibration signal and compares it with the preset sound emission requirements (i.e., Figure 5 (Preliminary verification of user intent). When the vibration signal is determined to meet the preset sound emission requirements, the aforementioned device initiates the extraction of audio features (i.e., Figure 5 (audio feature extraction in the process), and determine whether it meets the local processing requirements (i.e. Figure 5 (Whether it is processed locally).

[0061] Correspondingly, such as Figure 5 As shown, when local processing requirements are met, a lightweight semantic model (i.e., the first recognition model mentioned above) is used, and visual interaction commands are output to the earphones, which then perform the corresponding operations. When local processing requirements are not met, the voice is sent to the APP's large model (i.e., the second recognition model mentioned above), which outputs visual interaction commands to the earphones, which then perform the corresponding operations.

[0062] Furthermore, to achieve more accurate voice control, the step of determining whether the voice command meets the local processing requirements when the vibration signal meets the preset sound emission requirements includes: Step S1021: Perform time-domain alignment on the vibration signal and the voice command, and extract features from the aligned vibration signal to obtain the current vibration waveform, current vibration frequency, and current vibration intensity; Step S1022: Determine whether the vibration signal meets the preset voice-accompanied vibration requirements based on the current vibration waveform, the current vibration frequency, the current vibration intensity, and the aligned voice command; Step S1023: If yes, then determine whether the voice command meets the local processing requirements.

[0063] It can be understood that the above-mentioned time-domain alignment may refer to a processing method of synchronizing the vibration signal and the voice command in time, so as to achieve precise synchronization on the time axis and ensure that the corresponding moments of the two signals in subsequent analysis reflect the same event. The above-mentioned current vibration waveform may be the specific shape of the aligned vibration signal in the time domain. The above-mentioned current vibration frequency may be the main frequency value of the periodic component in the aligned vibration signal.

[0064] In addition, the above-mentioned current vibration intensity may be a measure of the energy or amplitude of the aligned vibration signal. The above-mentioned preset voice-associated vibration requirement may be a set of more refined comprehensive judgment criteria for determining whether the vibration signal is indeed accompanied by and originated from the user's voice, which may be more stringent than the preset voice generation requirement. For example, it is required that there is a specific correlation or matching relationship between the waveform, frequency and intensity characteristics of the vibration signal and certain acoustic characteristics (such as fundamental frequency, energy envelope) of the corresponding aligned voice command.

[0065] In a specific implementation, the above-mentioned device first performs cross-correlation analysis on the two-channel data of the collected vibration signal and voice command, calculates and compensates their time offset to achieve precise time-domain alignment. Subsequently, the above-mentioned device extracts the current vibration waveform (such as zero-crossing rate) characterizing its pattern, the current vibration frequency (such as fundamental frequency) reflecting its periodicity, and the current vibration intensity representing its energy magnitude from the aligned vibration signal through short-time analysis.

[0066] Next, the above-mentioned device performs multi-modal correlation analysis, calculates the cross-correlation coefficient of the vibration and the voice energy envelope to verify the intensity synchronization; checks whether the current vibration frequency is within the reasonable range of the fundamental frequency of human voice; compares the current vibration waveform characteristics with the preset voice vibration template; and confirms whether the vibration activity interval and the voice valid segment highly overlap in time. The above-mentioned device determines whether the vibration signal meets the preset voice-associated vibration requirement according to the comprehensive judgment result of the above conditions. If the comprehensive judgment is satisfied, the above-mentioned device confirms that this voice command originated from a valid user voice, and then judges whether it meets the local processing requirement.

[0067] For the sake of easy understanding, the following is illustrated by way of example, but no specific limitation is imposed on this embodiment: When the user wears headphones and says the "take a photo" command, the device first precisely aligns the "take a photo" audio collected by the microphone with the mandibular vibration signal collected by the accelerometer through cross-correlation analysis. Subsequently, the device extracts its characteristics from the aligned vibration signal: the waveform presents a double-peak pattern (current vibration waveform) corresponding to the syllables "take" and "photo", the main frequency is 120 Hz (current vibration frequency), and the intensity envelope is synchronized with the audio energy change.

[0068] Next, the device performs multimodal judgment: the cross-correlation coefficient of the energy envelopes of the two reaches 0.85 (higher than the threshold of 0.7); the vibration frequency of 120Hz is within the preset 80-300Hz voice fundamental frequency range; the vibration waveform characteristics match the voice vibration template with a degree of over 90%; and the vibration activity time completely covers the voice segment of "take a picture". All conditions meet the preset requirements for voice-accompanied vibration, so the device determines that the command is a valid voice output by the user and proceeds to the next step, analyzing that the short command "take a picture" matches the local keyword library, and finally triggering a local fast photo-taking response.

[0069] Based on the first and second embodiments, in the third embodiment, the content that is the same as or similar to that in Embodiments 1 and 2 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 6 , Figure 6 This is a flowchart of the third embodiment of the voice control method proposed in this application. Further, in order to obtain accurate first visual interaction commands, noise suppression can be performed using the collected vibrations. The step of recognizing the voice command through the first recognition model to generate the first visual interaction command includes: Step S21: Generate a reference noise spectrum based on the aligned vibration signal; Step S22: Adaptively filter the speech command using the reference noise spectrum; Step S23: Recognize the filtered voice command through the first recognition model to generate the first visual interaction command.

[0070] Understandably, the aforementioned reference noise spectrum can be a noise spectrum generated based on the aligned vibration signal, used to characterize the desired clean speech spectral properties. The aforementioned adaptive filtering can be a filtering technique that uses the reference noise spectrum as prior knowledge or guidance to dynamically estimate and suppress the spectral components in the speech command that correspond to environmental noise. The aforementioned filtered speech command can refer to an audio signal that has undergone adaptive filtering, where the target speech components are enhanced while the environmental noise components are suppressed.

[0071] In its implementation, the device first performs spectral analysis on the time-domain aligned vibration signal to generate a reference noise spectrum reflecting the distribution of user speech characteristics. Then, using this reference noise spectrum as a guide, the spectrum of the original speech command is estimated and adjusted, and an adaptive filtering algorithm is used to suppress frequency components unrelated to the reference spectrum. Finally, the device inputs the filtered enhanced speech signal into a first recognition model for recognition and generates the corresponding first visual interaction command.

[0072] Furthermore, in order to generate a reference noise spectrum, the step of generating a reference noise spectrum based on the aligned vibration signal includes: Step S211: Perform frame segmentation on the aligned vibration signal and windowing on the frame segmentation vibration signal; Step S212: Determine the short-time energy of each frame in the windowed vibration signal, select the short-time energy corresponding to the preset number of frames, and determine the energy threshold based on the short-time energy corresponding to the preset number of frames; Step S213: Perform threshold judgment between the energy threshold and the short-time energy of each frame to obtain the voice activity frame; Step S214: Perform a Fourier transform on each of the speech activity frames, and calculate the power spectral density based on each of the transformed speech activity frames to obtain the vibration power spectrum. Step S215: Generate a reference noise spectrum based on the vibration power spectrum of each vibration.

[0073] It is understandable that the aforementioned short-time energy can be the sum of the squares of the amplitudes of all sampling points within a windowed frame of signal, used to represent the intensity of that frame's signal. The aforementioned preset number of frames can be a pre-set value, used to specify the number of frames participating in the energy threshold calculation. The aforementioned energy threshold can be a threshold value dynamically calculated based on the short-time energy of the selected preset number of frames, used to distinguish between active and inactive speech frames. The aforementioned active speech frame can refer to a signal frame whose short-time energy exceeds the energy threshold. The aforementioned Fourier transform can be a mathematical transformation that converts the time-domain active speech frame signal to the frequency domain. The aforementioned power spectral density can be the distribution of signal power in the frequency domain, obtained by calculating the square of the amplitude of the Fourier transform result. The aforementioned vibration power spectrum can refer to the power spectral density result of the active speech frame.

[0074] In its implementation, the device first performs framing and windowing on the aligned vibration signal. Then, it calculates the short-time energy of each frame and dynamically determines an energy threshold based on the energy values ​​of a preset number of frames. By comparing the short-time energy of each frame with this threshold, the device filters out active speech frames. It then performs a Fourier transform on each active speech frame and calculates its power spectral density to obtain the corresponding vibration power spectrum. Finally, based on the vibration power spectra of all active speech frames, the device generates the final reference noise spectrum.

[0075] For ease of understanding, the following example is used for illustration, but it does not impose specific limitations on this embodiment. The device divides a 1-second vibration signal into frames and windows with a frame length of 20ms and a frame shift of 10ms, resulting in approximately 100 frames. After calculating the short-time energy of each frame, the median energy of the first 10 frames (assuming a silent segment) is multiplied by a coefficient of 1.5 as the energy threshold. Subsequently, all frames are traversed, and frames with energy exceeding the threshold (such as frames 30 to 65, corresponding to the vocal segment) are marked as active speech frames. FFT is performed on these active frames, and the power spectrum is calculated. Then, the power spectra of all active frames are averaged, smoothed, and normalized to finally generate a reference noise spectrum that characterizes the typical vibration spectrum features of the user during this vocalization.

[0076] Further, the step of generating a reference noise spectrum based on each of the vibration power spectra includes: Step S2151: Weight each vibration power spectrum using a preset frequency band weighting mapping function; Step S2152: Perform time averaging on each of the weighted vibration power spectra, and perform frequency domain smoothing on each of the time-averaged vibration power spectra; Step S2153: Normalize the frequency-smoothed power spectrum of each vibration to obtain the reference noise spectrum.

[0077] Understandably, the aforementioned preset frequency band weighting mapping function can be a predefined function that maps frequency points to weighting coefficients. Its design aims to enhance speech-related frequency bands and suppress irrelevant or noise-dominated frequency bands based on prior knowledge (such as the main distribution bands of speech formants and the transmission characteristics of vibration signals at different frequencies). The aforementioned weighting can refer to multiplying the value of each frequency point in the vibration power spectrum by the corresponding weighting coefficient. The aforementioned time averaging can refer to calculating the average or median of the weighted vibration power spectra across multiple (speech activity) frames in the time dimension. The aforementioned frequency domain smoothing can be a filtering operation performed on the frequency axis of the time-averaged power spectrum to eliminate random fluctuations and sharp spikes, resulting in a smoother spectral shape. The aforementioned normalization can be a proportional scaling of all values ​​in the frequency-smoothed power spectrum, making their maximum value 1 (or their sum equal to 1, etc.), thereby obtaining a reference template characterizing the relative spectral shape.

[0078] In practical use, the aforementioned device uses a preset frequency band weighting mapping function to weight the vibration power spectrum of each speech activity frame in the frequency dimension. Next, the device averages (or takes the median) all weighted vibration power spectra in the time dimension, and then smooths the single power spectrum obtained after time averaging on the frequency axis. Finally, the device normalizes and scales the smoothed power spectrum to obtain the final reference noise spectrum used to guide subsequent adaptive filtering.

[0079] For ease of understanding, the following example is provided, but it does not impose specific limitations on this embodiment. After obtaining the vibration power spectrum of multiple speech activity frames, the device applies a weighting function. This function assigns a weight of 1.2 to the 300-800Hz range (typical range of the first formant) and 1000-2000Hz range (energy of the second and third formants), and a weight of 0.3 to the ranges below 100Hz and above 2500Hz, in order to enhance the speech feature frequency bands. Subsequently, the median value of all weighted spectra is taken to obtain the average spectrum. Then, a five-point moving average is used to smooth the average spectrum in the frequency domain to remove glitches. Finally, the maximum value of the smoothed spectrum is normalized to 1 to obtain a reference noise spectrum that is relatively convex and smooth in the key speech segment, which is used to extract speech components with similar spectral characteristics from noisy audio.

[0080] This embodiment also provides a first embodiment of a voice control device, please refer to... Figure 7 , Figure 7 The diagram shows a voice control device provided in an embodiment of this application. The voice control device includes: The judgment module 701 is used to determine whether the voice command meets the local processing requirements when a voice command is received, wherein the local processing requirements are the requirements that the first recognition model supports for recognition. The first recognition module 702 is used to, if satisfied, recognize the voice command through the first recognition model, generate a first visual interaction command, and execute the operation corresponding to the first visual interaction command. The second recognition module 703 is used to upload the voice command to the cloud device if the conditions are not met, receive the second visual interaction command sent by the cloud, and execute the operation corresponding to the second visual interaction command. The second visual interaction command is generated by recognizing the voice command through the second recognition model. The judgment module 701 is further configured to remove silent segments from the voice command, determine the duration of the voice command after removal, and determine whether the duration is not higher than a preset duration threshold and / or determine the text length of the voice command, estimate the number of words based on the text length, and determine whether the number of words is not higher than a preset number threshold; and / or determine the current keyword in the voice command and determine whether the current keyword is a preset keyword.

[0081] Referring to the first embodiment of the voice control device, this embodiment also proposes a second embodiment of the voice control device. The contents that are the same as or similar to those in the first embodiment of the voice control device can be referred to the above description, and will not be repeated hereafter.

[0082] The judgment module 701 is further configured to, upon receiving a voice command, use the vibration sensing component to sense vibrations in the wearer and obtain a vibration signal; and if the vibration signal meets the preset sound requirements, determine whether the voice command meets the local processing requirements. The judgment module 701 is further configured to perform time-domain alignment on the vibration signal and the voice command, and to extract features from the aligned vibration signal to obtain the current vibration waveform, the current vibration frequency, and the current vibration intensity; and to determine whether the vibration signal meets the preset voice-accompanied vibration requirements based on the current vibration waveform, the current vibration frequency, the current vibration intensity, and the aligned voice command; if so, to determine whether the voice command meets the local processing requirements.

[0083] Referring to the first embodiment and the second embodiment of the voice control device, this embodiment also proposes a third embodiment of the voice control device. The contents that are the same as or similar to the first embodiment and the second embodiment of the voice control device can be referred to the above description, and will not be repeated hereafter.

[0084] The first recognition module 702 is further configured to generate a reference noise spectrum based on the aligned vibration signal; adaptively filter the voice command using the reference noise spectrum; and recognize the filtered voice command through the first recognition model to generate a first visual interaction command. The first recognition module 702 is further configured to perform frame segmentation processing on the aligned vibration signal, and windowing processing on the frame segmentation vibration signal; determine the short-time energy of each frame in the windowed vibration signal, and select the short-time energy corresponding to a preset number of frames, and determine an energy threshold based on the short-time energy corresponding to the preset number of frames; perform threshold judgment between the energy threshold and the short-time energy of each frame to obtain a speech activity frame; perform Fourier transform on each speech activity frame, and calculate the power spectral density based on each transformed speech activity frame to obtain a vibration power spectrum; and generate a reference noise spectrum based on each vibration power spectrum. The first identification module 702 is further configured to weight each vibration power spectrum using a preset frequency band weighting mapping function; perform time averaging on each weighted vibration power spectrum and frequency domain smoothing on each time-averaged vibration power spectrum; and normalize each frequency-domain smoothed vibration power spectrum to obtain a reference noise spectrum.

[0085] The voice control device provided in this embodiment, employing the voice control method described in the above embodiments, can solve the technical problem of how to improve the user experience during voice control. Compared with the prior art, the beneficial effects of the voice control device provided in this embodiment are the same as those of the voice control method provided in the above embodiments, and other technical features in the voice control device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0086] This embodiment provides a smart headset, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the voice control method in Embodiment 1 above.

[0087] The following is for reference. Figure 8 , Figure 8 This is a schematic diagram of the structure of a smart earphone suitable for implementing the embodiments of this application. Figure 8 The smart earphones shown are merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0088] like Figure 8 As shown, the smart headset includes a first recognition model and a vibration sensing component, as well as a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the smart headset. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the smart headset to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows smart headsets with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0089] Specifically, according to this embodiment, the process described above with reference to the flowchart can be implemented as a computer software program. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the disclosed embodiments of this embodiment.

[0090] The smart earphone provided in this embodiment employs the voice control method described in the above embodiments, which can solve the technical problem of how to improve the user experience during voice control. Compared with the prior art, the beneficial effects of the smart earphone provided in this embodiment are the same as those of the voice control method provided in the above embodiments, and other technical features of the smart earphone are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0091] It should be understood that the various parts disclosed in this embodiment can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0092] The above description is merely a specific implementation of this embodiment, but the protection scope of this embodiment is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this embodiment should be included within the protection scope of this embodiment. Therefore, the protection scope of this embodiment should be determined by the protection scope of the claims.

[0093] This embodiment provides a computer-readable voice control system having computer-readable program instructions (i.e., computer programs) stored thereon, which are used to execute the voice control method in the above embodiment.

[0094] The computer-readable voice control system provided in this embodiment can be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of a computer-readable voice control system may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable voice control system can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable voice control system can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0095] The aforementioned computer-readable voice control system may be included in the smart headset; or it may exist independently and not be assembled into the smart headset.

[0096] The aforementioned computer-readable voice control system carries one or more programs, which, when executed by the smart earphone, enable the smart earphone to perform voice control.

[0097] Computer program code for performing the operations of this embodiment can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this embodiment. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0099] The modules described in this embodiment can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0100] The readable voice control system provided in this embodiment is a computer-readable voice control system. This system stores computer-readable program instructions (i.e., computer programs) for executing the aforementioned voice control method, and can solve the technical problem of how to improve the user experience during voice control. Compared with the prior art, the beneficial effects of the computer-readable voice control system provided in this embodiment are the same as those of the voice control method provided in the above embodiments, and will not be repeated here.

[0101] The above descriptions are only some embodiments and do not limit the patent scope of this embodiment. All equivalent structural transformations made based on the technical concept of this application and the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the patent protection scope of this application.

Claims

1. A voice control method, characterized in that, The method is applied to a smart earphone in a voice control system, wherein a first recognition model is deployed in the smart earphone. The voice control system further includes: a cloud device, in which a second recognition model is deployed, wherein the model size of the first recognition model is smaller than that of the second recognition model, and the smart earphone is connected to the cloud device; The method includes: Upon receiving a voice command, it is determined whether the voice command meets the local processing requirements, which are the requirements that the first recognition model supports for recognition. If the conditions are met, the voice command is recognized by the first recognition model, a first visual interaction command is generated, and the operation corresponding to the first visual interaction command is executed. If the conditions are not met, the voice command is uploaded to the cloud device, the second visual interaction command sent by the cloud is received, and the operation corresponding to the second visual interaction command is executed. The second visual interaction command is generated by recognizing the voice command through the second recognition model.

2. The method as described in claim 1, characterized in that, The step of determining whether the voice command meets the local processing requirements includes: The voice command is muted, the duration of the muted segment is determined, and it is determined whether the duration is not higher than a preset duration threshold. And / or, determine the text length of the voice command, estimate the number of words based on the text length, and determine whether the number of words is not higher than a preset number threshold; And / or, determine the current keyword in the voice command, and determine whether the current keyword is a preset keyword.

3. The method as described in claim 1, characterized in that, The smart earphones are also equipped with vibration sensing components; The step of determining whether a received voice command meets local processing requirements includes: Upon receiving a voice command, the vibration sensing component detects vibrations from the wearer and obtains vibration signals. If the vibration signal meets the preset sound emission requirements, determine whether the voice command meets the local processing requirements.

4. The method as described in claim 3, characterized in that, The step of determining whether the voice command meets the local processing requirements when the vibration signal meets the preset sound emission requirements includes: The vibration signal and the voice command are time-domain aligned, and the aligned vibration signal is feature-extracted to obtain the current vibration waveform, current vibration frequency, and current vibration intensity. Based on the current vibration waveform, the current vibration frequency, the current vibration intensity, and the aligned voice command, determine whether the vibration signal meets the preset voice-accompanied vibration requirements; If so, determine whether the voice command meets the local processing requirements.

5. The method as described in claim 4, characterized in that, The step of recognizing the voice command through the first recognition model and generating the first visual interaction command includes: A reference noise spectrum is generated based on the aligned vibration signal; The speech command is adaptively filtered using the reference noise spectrum. The first recognition model recognizes the filtered voice commands and generates the first visual interaction command.

6. The method as described in claim 5, characterized in that, The step of generating a reference noise spectrum based on the aligned vibration signal includes: The aligned vibration signal is framed, and then windowed. Determine the short-time energy of each frame in the windowed vibration signal, select the short-time energy corresponding to a preset number of frames, and determine the energy threshold based on the short-time energy corresponding to the preset number of frames. The energy threshold is compared with the short-time energy of each frame to obtain the voice activity frame; Perform a Fourier transform on each of the aforementioned speech activity frames, and calculate the power spectral density based on each transformed speech activity frame to obtain the vibration power spectrum. A reference noise spectrum is generated based on the vibration power spectrum of each vibration.

7. The method as described in claim 6, characterized in that, The step of generating a reference noise spectrum based on each of the vibration power spectra includes: Each vibration power spectrum is weighted by a preset frequency band weighting mapping function; The time-averaged spectrum of each weighted vibration power spectrum is then smoothed in the frequency domain. The power spectrum of each vibration after frequency domain smoothing is normalized to obtain the reference noise spectrum.

8. A voice control device, characterized in that, The device includes: The judgment module is used to determine whether the voice command meets the local processing requirements when a voice command is received, wherein the local processing requirements are the requirements that the first recognition model supports for recognition. The first recognition module is configured to, if satisfied, recognize the voice command through the first recognition model, generate a first visual interaction command, and execute the operation corresponding to the first visual interaction command. The second recognition module is used to upload the voice command to the cloud device if the conditions are not met, receive the second visual interaction command sent by the cloud, and execute the operation corresponding to the second visual interaction command. The second visual interaction command is generated by recognizing the voice command through the second recognition model.

9. A smart earphone, characterized in that, The smart earphone has a first recognition model deployed inside it; The smart earphone further includes: a memory, a processor, and a voice control program stored in the memory and executable on the processor, wherein the voice control program, when executed by the processor, implements the steps of the voice control method as described in any one of claims 1 to 7.

10. A voice control system, characterized in that, The voice control system includes: a cloud device and the smart earphone as described in claim 9; The cloud device has a second recognition model deployed within it. The model size of the first recognition model is smaller than that of the second recognition model. The smart earphones are connected to the cloud device.

Images