Speech processing methods, processors and electronic devices

Abstract

This application provides a voice processing method, processor, and electronic device, relating to the field of terminal technology. When a second processor detects that a voice interaction command includes wake-up audio, it sends a wake-up signal to a first processor. The first processor responds to the input wake-up signal, enters a wake-up state, and receives the control audio included in the voice interaction command. Since the first processor only enters the wake-up state after receiving the wake-up signal, its power consumption is reduced, thereby reducing the overall power consumption of the electronic device containing the first processor, increasing the battery life of the electronic device, and improving the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of terminal technology, and in particular to a voice processing method, processor, and electronic device. Background Technology

[0002] With the continuous development of voice interaction technology, users can input voice control commands to electronic devices while simultaneously waking them up. This voice interaction method, which combines wake-up and control, reduces the waiting time for users when interacting with electronic devices via voice, thus improving interaction efficiency.

[0003] To achieve the above-mentioned voice interaction method, the processor of the electronic device usually needs to be kept in a wake-up state at a high operating frequency so that when the wake-up keyword included in the user's voice control command is captured, the control audio included in the voice control command can be responded to immediately.

[0004] However, keeping the processor of an electronic device constantly awake at a high operating frequency increases the overall power consumption of the device, reduces its battery life, and negatively impacts the user experience. Summary of the Invention

[0005] This disclosure provides a voice processing method, processor, and electronic device that can solve the problem of the processor of an electronic device always maintaining a wake-up state at a high operating frequency, which increases the overall power consumption of the electronic device.

[0006] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, this disclosure provides a speech processing method applied to a first processor, the method comprising: When in a non-wake-up state, it enters a wake-up state in response to an input wake-up signal; the wake-up signal is emitted by the second processor upon detecting a voice interaction command including wake-up audio; in response to entering the wake-up state, it receives control audio including a voice interaction command.

[0007] Based on the above technical solution, the present invention can be further improved as follows.

[0008] Furthermore, in response to entering the wake-up state, the system directly receives control audio including voice interaction commands; and / or, in response to entering the wake-up state, the system receives spliced ​​audio sent by the second processor; the spliced ​​audio is obtained by the second processor performing splicing processing on the wake-up audio and the control audio.

[0009] Furthermore, when directly receiving control audio included in voice interaction commands, the wake-up audio and control audio are spliced ​​together to obtain spliced ​​audio.

[0010] Furthermore, a voice separation operation is performed on the spliced ​​audio to obtain the target audio.

[0011] Furthermore, the target audio is sent to the server so that the server can parse the semantic information corresponding to the target audio, and the server can send control commands to the target device based on the semantic information corresponding to the target audio.

[0012] Furthermore, the target device includes at least one of the following: The first device; the first device includes a first processor; Associated device; the associated device has a direct or indirect communication connection with the first device.

[0013] Furthermore, obtain the semantic information corresponding to the target audio; generate control commands based on the semantic information corresponding to the target audio; execute the control commands, or send control commands to associated devices.

[0014] Furthermore, the associated equipment includes at least one of the following: The second device; the second device has a direct communication connection with the first device; the first device includes a first processor; Third device: The third device has a direct communication connection with the second device.

[0015] Furthermore, when the associated device is a second device, a control command is sent to the second device to cause the second device to execute the control command; when the associated device is a third device, a control command is sent to the second device to cause the second device to forward the control command to the third device, and to cause the third device to execute the control command.

[0016] Furthermore, the second or third device is an in-vehicle infotainment system included in the vehicle; the control commands include at least one of the following: Control commands used to trigger the information collection components included in the vehicle to perform in-vehicle information collection tasks; Control commands used to trigger the audio playback components included in the vehicle to perform audio playback tasks; Control commands are used to trigger adjustable components of the vehicle to perform adjustment tasks; the adjustable components include at least one of windows, wipers, seats, doors, lighting components, display components, and air conditioning components.

[0017] Furthermore, play the response audio corresponding to the voice interaction command.

[0018] Furthermore, the end point of the wake-up audio time acquisition is the starting point of the control audio time acquisition; the duration of the wake-up audio is the target duration.

[0019] Furthermore, both the first processor and the second processor are located in the first device, and the power consumption of the second processor is less than that of the first processor.

[0020] Furthermore, in response to entering the wake-up state, a first number of audio acquisition components among the multiple audio acquisition components are controlled to be in the running state; the first number is greater than the second number; the second number is the number of audio acquisition components in the running state among the multiple audio acquisition components when the first processor is in the non-wake-up state.

[0021] Furthermore, in response to the input wake-up signal, the operating frequency is switched from the first operating frequency to the second operating frequency; the second operating frequency is greater than the first operating frequency.

[0022] Secondly, this disclosure provides a speech processing method applied to a second processor, the method comprising: In response to detecting a voice interaction command including wake-up audio, a wake-up signal is sent to a first processor to cause the first processor to enter a wake-up state in response to the input wake-up signal, and to cause the first processor to receive control audio included in the voice interaction command while in the wake-up state.

[0023] Thirdly, this disclosure provides a processor, including: a memory and a controller; the memory and the controller are coupled; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the controller, cause the processor to perform the steps in the speech processing method described in the first aspect and any one thereof.

[0024] Fourthly, this disclosure provides a processor, including: a memory and a controller; the memory and the controller are coupled; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the controller, cause the processor to perform the steps in the speech processing method described in the second aspect and any one thereof.

[0025] Fifthly, this disclosure provides an electronic device including the processor described in the third aspect above, and / or including the processor described in the fourth aspect above.

[0026] The voice processing method, processor, and electronic device provided in this disclosure enable a second processor to continuously detect whether the user's input voice interaction command includes wake-up audio, and send a wake-up signal to the first processor when wake-up audio is detected. The first processor can respond to the wake-up signal, enter a wake-up state, and receive the control audio included in the voice interaction command. Since the first processor does not need to detect wake-up audio, it does not need to remain in a wake-up state continuously; instead, it only enters a wake-up state after receiving a wake-up signal. This reduces the power consumption of the first processor, thereby reducing the overall power consumption of the electronic device in which the first processor resides, increasing the battery life of the electronic device, and improving the user experience. Attached Figure Description

[0027] Figure 1 A flowchart illustrating the first speech processing method provided in this application embodiment; Figure 2 A schematic diagram illustrating the processing flow of the wake-up audio and control audio by the first processor provided in an embodiment of this application; Figure 3 A flowchart illustrating the second speech processing method provided in this application embodiment; Figure 4 A flowchart illustrating the third speech processing method provided in this application embodiment; Figure 5 This application provides a schematic diagram of the structure of a server according to an embodiment of the present application. Figure 6 This application provides a schematic diagram of a communication connection between an electronic device and a server. Figure 7 This is a schematic diagram of a vehicle provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes.

[0029] With the continuous development of voice interaction technology, voice interaction functions of electronic devices have become a common feature in people's daily lives. Users can interact with electronic devices by voice to achieve their corresponding functions.

[0030] In one related technology, a user can input a wake-up audio signal to an electronic device to activate it. Once activated, the user can then input control audio signals to instruct the device to perform corresponding functions. However, this distributed execution of wake-up and control results in a significant waiting period between activating the device via voice and controlling it via voice. This not only lengthens the entire human-computer interaction cycle but also increases the cumbersomeness of voice interaction steps, making it difficult to meet users' core need for efficient and convenient voice interaction.

[0031] In another related technology, the electronic device can always be in a wake-up state. In this case, the user can input voice interaction commands to the electronic device, including wake-up audio and control audio. After detecting that the user's voice interaction command includes the wake-up audio, the electronic device can directly respond to the control audio included in the voice interaction command. For example, while the electronic device is in a wake-up state, the user can input the voice interaction command "[User's name], I want to listen to music." After detecting the wake-up audio "[User's name]," the electronic device will execute the music playback operation based on the acquired control audio "[User's name], I want to listen to music."

[0032] However, this simultaneous wake-up and control execution mode requires the electronic device to be constantly awake, meaning its processor needs to run at a high frequency continuously to monitor and process user-input voice commands in real time. This leads to a significant increase in the overall power consumption of the electronic device. For electronic devices, especially wearable devices with limited battery capacity such as smartwatches and AR glasses, high power consumption shortens battery life, requiring frequent charging and negatively impacting the user experience.

[0033] To address the aforementioned issues, embodiments of this application provide a voice processing method, apparatus, electronic device, and vehicle, enabling a second processor to continuously detect whether a user-input voice interaction command includes wake-up audio, and to send a wake-up signal to a first processor upon detection of wake-up audio. The first processor, in response to the wake-up signal, enters a wake-up state and receives control audio included in the voice interaction command.

[0034] Since the first processor does not need to detect the wake-up audio, it does not need to remain in a wake-up state at all times. It only enters the wake-up state after receiving a wake-up signal, which can effectively reduce the power consumption of the first processor, thereby reducing the overall power consumption of the electronic device in which it is located (which can also be referred to as the first device in this embodiment of the application), increasing the battery life of the first device, and improving the user experience.

[0035] Furthermore, in related technologies, integrated processors used for voice processing have relatively slow startup and operation speeds due to their large capacity and large workload, and their response to wake-up and control audio is often slow. In contrast, in this embodiment, the second processor is used as a lightweight processing unit, which can continuously and quickly detect whether the user's voice interaction command includes wake-up audio. Upon detecting wake-up audio, it immediately sends a wake-up signal to the first processor, avoiding the first processor from increasing its startup burden by undertaking the wake-up detection task itself. This effectively solves the problems of slow operation and easy errors due to complex processing logic of integrated processors in related technologies.

[0036] Furthermore, in this embodiment, the second processor can continue to capture subsequent control audio after detecting the wake-up audio, thus preserving the coherent speech stream of the wake-up audio and control audio. While sending the wake-up signal to the first processor, it can also transmit the audio spliced ​​together or the complete control audio to the first processor, ensuring the integrity of the speech splicing. This allows the first processor to be woken up without waiting for additional audio splicing steps and to immediately carry out subsequent processing based on the complete audio, significantly improving the overall speech processing efficiency.

[0037] Meanwhile, this application embodiment also solves the problem in traditional voice processing solutions where the integrated processor included in the device cannot recognize the control audio when the user continuously inputs wake-up audio and control audio: Since the second processor is always in voice detection state, when the user continuously outputs voice interaction commands including wake-up audio and control audio, the second processor can capture the complete voice stream in real time; after the first processor is quickly woken up by the second processor, it can directly receive the complete audio transmitted by the second processor or continue to receive subsequent control audio, which not only realizes the rapid wake-up of the first processor, but also ensures the accurate recognition of continuous voice interaction commands, and will not result in the inability to recognize control audio due to continuous input of commands.

[0038] Finally, in this embodiment, the second processor can perform wake-up detection and pre-audio processing tasks. After the first processor is woken up, it only needs to focus on controlling the reception and subsequent processing of audio. The clear division of labor between the two further improves the response speed to voice interaction commands, and ultimately achieves a fast response to voice interaction commands, allowing users to have a smooth and lag-free user experience.

[0039] Figure 1This is a flowchart illustrating a first speech processing method provided in an embodiment of this application. The first speech processing method provided in this application can be applied to a first processor. That is, the executing entity of the first speech processing method provided in this application can be a first processor.

[0040] The first processor may be a processor included in the first device. The first device may be a device such as a mobile phone, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), handheld computer, netbook, personal digital assistant (PDA), wearable electronic device, virtual reality device, in-vehicle system, etc. Furthermore, wearable devices may be devices such as smart glasses, smartwatches, and smart bracelets, etc., and this application embodiment does not impose any limitations on this.

[0041] like Figure 1 As shown, the first speech processing method provided in this application embodiment may include the following steps: S101: When the first processor is in a non-wake-up state, it enters a wake-up state in response to the input wake-up signal.

[0042] The wake-up signal is emitted by the second processor upon detecting a voice interaction command, including a wake-up audio signal.

[0043] Furthermore, the first processor is the processing unit in the first device that undertakes the main data processing tasks, has high computing power, and consumes relatively large amounts of energy.

[0044] The second processor can be a processing unit for voice wake-up detection, capable of continuously monitoring whether the external voice input includes wake-up audio.

[0045] During the period when the first processor is in a non-wake-up state, the second processor can capture external voice signals in real time and analyze the captured voice signals to determine whether the captured voice signals are the voice signals corresponding to the wake-up audio. If it is determined that the captured voice signals are the voice signals corresponding to the wake-up audio, a wake-up signal can be sent to the first processor. After receiving the wake-up signal, the first processor can switch from the non-wake-up state to the wake-up state in response to the received wake-up signal.

[0046] Specifically, wake-up audio refers to voice signals that include wake-up keywords, such as the voice signals corresponding to preset wake-up keywords like "hello" or "xx classmate".

[0047] It's important to note that wake-up audio includes the audio signal corresponding to the wake-up keyword, not just the audio signal corresponding to the wake-up keyword. For example, in "Student XX, I want to listen to music," "Student XX" is the wake-up keyword. In this case, "Student XX," "Student XX, I," and "Student XX, I want to listen to music" can all be wake-up audio.

[0048] S102: The first processor, in response to entering the wake-up state, receives control audio including voice interaction commands.

[0049] Specifically, the voice interaction commands output by the user can include wake-up audio and control audio. The description of the wake-up audio can be found in the preceding embodiments and will not be repeated here. Furthermore, the control audio is the audio output after the user outputs the wake-up audio. For example, in the command "Student XX, I want to listen to music," if "Student XX" is the wake-up audio, then "I want to listen to music" is the control audio. Or, if "Student XX, I" is the wake-up audio, then "I want to listen to music" is the control audio, and so on.

[0050] As can be seen, the first voice processing method provided in this application establishes a highly efficient and low-power voice interaction mechanism between the user and the first device through the division of labor and cooperation between the first processor and the second processor. Specifically, the second processor continuously monitors the wake-up audio, avoiding the energy waste caused by the high-computing-power first processor remaining in a wake-up state for extended periods, significantly improving the battery life of the first device, and is particularly suitable for energy-sensitive wearable device scenarios such as smart glasses.

[0051] Furthermore, thanks to the seamless integration of wake-up audio and control audio, the first processor can quickly wake up upon receiving a wake-up signal and rapidly receive control commands, including those for voice interaction. This not only avoids the cumbersome process of users having to re-enter control voice commands after waking up the corresponding device, as is common in traditional wake-up mechanisms, but also improves the response speed of the corresponding device to voice interaction commands, thus enhancing the user experience.

[0052] Additionally, in some embodiments, see Figure 2 In response to entering the wake-up state, the first processor directly receives the control audio included in the voice interaction command; and / or, in response to entering the wake-up state, receives the spliced ​​audio sent by the second processor; the spliced ​​audio is obtained by the second processor performing splicing processing on the wake-up audio and the control audio.

[0053] The following is an exemplary description of the process by which the first processor directly receives voice interaction commands, including control audio.

[0054] After the second processor detects the wake-up audio and triggers the first processor to switch from a non-wake-up state to a wake-up state, the first processor can directly obtain the control audio included in the voice interaction command. For example, when the user utters the voice interaction command "Student xx, check the time" in sequence, the second processor detects "Student xx" (wake-up audio) and causes the first processor to enter the wake-up state. The first processor can then actively receive the control audio "check the time" included in the voice interaction command.

[0055] The first processor directly acquires the control audio without needing to be forwarded by the second processor, reducing intermediate steps in the transmission of control audio and enabling the first processor to quickly obtain the control audio output by the user, thus improving the response speed to voice interaction commands.

[0056] The following is an exemplary description of the process by which the first processor receives spliced ​​audio sent by the second processor.

[0057] After the second processor detects the wake-up audio, it can continue to receive control audio. For example, when the user utters the voice interaction command "Student XX, please check the time," the second processor detects "Student XX" (wake-up audio) and can continue to receive the control audio "check the time." Upon receiving the control audio, the second processor can concatenate the wake-up audio and control audio, sending a wake-up signal and the concatenated audio to the first processor. Responding to the received wake-up signal, the first processor switches from a non-wake-up state to a wake-up state and can then receive the concatenated audio sent by the second processor.

[0058] The second processor, upon detecting the wake-up audio, continues to receive control audio, enabling it to capture the continuous speech stream of both wake-up and control audio. This avoids situations where the first and second processors collect different types of audio (e.g., the first processor collects wake-up audio, and the second processor collects control audio after switching to the wake-up state). In such cases, the brief intervals during which the second processor sends a wake-up signal to the first processor, or the brief intervals during which the first processor switches from the non-wake-up state to the wake-up state, can cause the loss or breakage of speech segments at the junction of the wake-up and control audio, or result in repeated audio acquisition at the junction of the wake-up and control audio. This reduces the complexity of subsequent audio processing.

[0059] Furthermore, the embodiments of this application can take advantage of the low power consumption of the second processor to undertake the splicing task, thereby reducing the computing power burden of the first processor and reducing the energy consumption of the first processor, thus reducing the overall energy consumption of the first device, especially suitable for wearable devices that are sensitive to power consumption, such as smart glasses.

[0060] Additionally, in some embodiments, see also [link to previous document]. Figure 2in a case where the first processor directly receives the control audio included in the voice interaction instruction, the first processor performs splicing processing on the wake-up audio and the control audio to obtain a spliced audio.

[0061] The core of the splicing processing is to solve the problem that when a user outputs the wake-up audio and the control audio in a coherent manner at an excessively fast speech rate, the voice segment at the junction of the wake-up audio and the control audio is lost or broken, or the problem of repeated audio collection occurs at the junction of the wake-up audio and the control audio.

[0062] For example, when a user outputs the wake-up audio and the control audio quickly and coherently, the start portion of the control audio may be highly temporally connected to the end portion of the wake-up audio. If the two types of audio are acquired separately, the first character of the control audio is prone to be lost due to recognition delay or capture deviation, which in turn leads to breakage of the instruction semantics. By means of audio splicing, the wake-up audio and the control audio are integrated into a continuous audio whole, which can complete the possibly lost first character information of the control audio, ensure that the spliced audio completely retains the original semantics of the user instruction, and avoid parsing errors caused by the lack of partial information.

[0063] Taking a scenario where the electronic device is a smart glasses as an example, when a user performs voice interaction with the smart glasses, the user can coherently output a voice interaction instruction of "XX classmate, what is the weather today". If the user speaks too fast, the first character "今" (meaning "today") at the start of the control audio may be too tightly connected to the end of the wake-up word "XX classmate", resulting in the loss of the first character in the separately captured control audio, which is only recognized as "天天气怎么样" (meaning "what is the weather like today" with the first character repeated); after the first processor acquires the wake-up audio of "XX classmate 今", the first processor can splice the wake-up audio of "XX classmate 今" and the control audio of "what is the weather like today" to form a spliced audio of "XX classmate, what is the weather today".

[0064] In an implementation manner, the first processor may perform endpoint detection on the acquired wake-up audio and control audio, and identify the end endpoint of the wake-up audio and the start endpoint of the control audio. Since the wake-up audio and the control audio are voice signals output coherently by the user, the two are naturally connected on the time axis, and only a tiny gap or overlap may exist due to separate capture. The first processor can accurately align the end endpoint of the wake-up audio with the start endpoint of the control audio in chronological order, eliminate the gap or deduplicate the overlapping part, form a continuous and unbroken spliced audio, and ensure complete connection between the wake-up audio and the control audio.

[0065] It can be seen that, through the splicing processing of the wake-up audio and the control audio, the first processor accurately solves the problem that the first character of the control audio is lost due to the excessively fast speech rate of the user, so that the spliced audio completely retains the original semantics of the user instruction, and fundamentally avoids subsequent instruction parsing errors caused by the lack of partial information.

[0066] In addition, in some embodiments, continue to refer to Figure 2 The first processor can perform voice separation on the spliced ​​audio to obtain the target audio.

[0067] The core of the voice separation operation on the spliced ​​audio is to extract the included human voice audio as the target audio. The extracted human voice audio can include only the voice audio of the target user, for example, extracting only the voice audio of the user located in the driver's seat. Alternatively, the extracted human voice audio can include voice audio of users in different areas of the vehicle, for example, extracting the voice audio of users located in the driver's seat, front passenger seat, second-row left seat, and second-row right seat separately, so that corresponding processing can be performed on the voice audio of different users.

[0068] Specifically, the first processor can perform multi-dimensional analysis on the spliced ​​audio, including voiceprint features, frequency range, and time-domain waveform, to filter out signals that match the voice characteristics of the target user (the user issuing the voice interaction command) (such as the general human voice frequency range of 100Hz-10kHz, or voiceprint features matching the user's historical voice samples). Furthermore, it can identify non-human voice components in the spliced ​​audio based on a preset noise feature library (such as the frequency distribution and waveform characteristics of ambient noise and background noise). Then, audio separation algorithms (such as spectral mask-based separation algorithms) can be used to remove the identified noise components from the spliced ​​audio, while preserving and enhancing the target user's human voice audio signal, ultimately resulting in target audio containing only the target user's clear human voice.

[0069] For example, taking smart glasses as the first device and a noisy city street as the scenario, a user wearing smart glasses, amidst the noise of car horns and crowds, continuously outputs the voice interaction command "How's the weather today, classmate xx?". The second processor captures the wake-up audio of "classmate xx today," triggering the wake-up of the first processor. After receiving the control audio of "how's the weather today," the first processor can first perform endpoint detection on the two types of audio, "classmate xx today" and "how's the weather today," identifying a slight overlap between the end endpoint of "classmate xx today" and the start endpoint of "how's the weather today" (due to excessive speaking speed). Then, it precisely aligns the endpoints according to the chronological order, removes the overlapping parts, and eliminates potential gaps to form a continuous spliced ​​audio of "how's the weather today, classmate xx?". This spliced ​​audio not only completes the missing first word of the control audio but also maintains the semantic coherence of the command, although it is still mixed with various noises from the street environment.

[0070] Then, the first processor can use multi-dimensional analysis to filter out signals that match the user's voice characteristics (such as matching the user's historically stored voiceprint samples and being in the 100Hz-10kHz human voice frequency range); then, based on a preset noise feature library, it can identify non-human voice components in the spliced ​​audio, such as vehicle horns (high-frequency noise at a specific frequency) and crowd noise (irregular broad-spectrum noise); finally, through a separation algorithm based on spectral masking, these noise components are removed, while the clarity of the user's voice signal is enhanced, ultimately obtaining the target audio containing only a clear human voice, such as "Hey xx, how's the weather today?".

[0071] As can be seen, this embodiment of the application performs human voice separation on spliced ​​audio through multi-dimensional feature analysis and separation algorithms, removing noise to obtain a clean human voice target audio. This not only solves the core pain points of missing first characters and noise interference, improving the accuracy of subsequent instruction parsing, but also adapts to noisy scenarios and optimizes the user's interactive experience.

[0072] In addition, in some embodiments, the first processor may send target audio to the server so that the server can parse the semantic information corresponding to the target audio, and the server can send control commands to the target device based on the semantic information corresponding to the target audio.

[0073] Semantic information refers to the interpretation of the user's voice commands in the target audio, accurately reflecting the user's core needs. After receiving the target audio sent by the first processor, the server can first extract the effective speech components included in the target audio, providing a foundation for subsequent semantic parsing.

[0074] Specifically, to extract the effective speech components from the target audio and eliminate any residual weak noise or silent segments, the server can use the VAD (Voice Activity Detection) algorithm to process the target audio. For example, the server can use the VAD algorithm to analyze key features such as the energy value and frequency distribution of the target audio to distinguish between speech segments and non-speech segments, retaining only the user's effective speech and filtering out meaningless invalid components, allowing subsequent semantic parsing to focus solely on the core speech content.

[0075] Afterwards, the server can invoke a semantic recognition algorithm to perform deep semantic analysis on the valid speech segments processed by the VAD algorithm. The semantic recognition algorithm is used to comprehensively analyze the speech features, word combinations, grammatical structures, and semantic relationships in the valid speech segments, directly converting the user's speech signal into semantic information that accurately reflects the core needs, avoiding misjudgment of intent caused by redundant components in the speech signal.

[0076] Once the semantic parsing is complete, the server can generate corresponding control commands based on the extracted semantic information and send the control commands to the target device.

[0077] The control commands can clearly define the user's core requirements and embed the target device's unique identifier to ensure that the control commands can be accurately received by the target device.

[0078] Furthermore, to ensure transmission security and integrity, the server can issue control commands through a pre-agreed encrypted transmission protocol (such as HTTPS or MQTT) with the target device, and simultaneously synchronize a status notification to the first processor that "control commands have been issued to the target device," allowing the first processor to monitor the task progress in real time.

[0079] As can be seen, the embodiments of this application transfer computationally intensive operations such as audio purification, semantic parsing, and instruction generation to the server, which greatly reduces the local processing pressure of the first processor and avoids high energy consumption of the first device due to complex calculations. It is especially suitable for wearable devices with limited resources and sensitive to power consumption, such as smart glasses, and effectively extends their battery life.

[0080] In addition, in some embodiments, the target device includes at least one of the following: The first device; the first device includes a first processor; Associated device; the associated device has a direct or indirect communication connection with the first device.

[0081] The first device can be a wearable device that is power-sensitive and portable, such as smart glasses, which receives and performs preliminary processing of voice interaction commands through a first processor.

[0082] A direct communication connection refers to a data transmission link established directly between the first device and the associated device without the need for any other intermediate devices, through a preset communication protocol (such as Bluetooth, Wi-Fi, or automotive-grade Ethernet).

[0083] Indirect communication connection refers to a situation where the first device and the associated device do not directly establish a communication link, but need to rely on other devices (such as mobile phones, tablets, etc.) as intermediate data relay nodes to indirectly establish a data transmission link.

[0084] In conjunction with the foregoing embodiments, the server can send control commands to the target device.

[0085] When the first device is the target device, it can directly receive and execute control commands sent by the server, ensuring the immediacy of the control command execution.

[0086] For example, the first device is smart glasses. When a user speaks the voice interaction command "Student xx, please check the current time", the first processor obtains the target audio and sends it to the server. The server can then parse the semantic information corresponding to the target audio and generate a control command to represent "get and broadcast the current time". The server then sends the generated control command to the smart glasses (the first device). After receiving the control command, the first processor can activate the local time module to obtain real-time time data and then execute the control command through the audio playback module to output the voice information "the current time is 10:30 am".

[0087] When the associated device is used as the target device, it can also directly receive and execute control commands sent by the server, ensuring the immediacy of the control command execution.

[0088] For example, the first device is smart glasses, and the associated device is the vehicle's infotainment system. Both are in close proximity within the vehicle and establish a direct communication connection (such as via Bluetooth). The user utters a voice command, "Student XX, adjust the car's air conditioning to 24℃." The first processor receives the target audio and sends it to the server. The server then parses the semantic information to generate a control command representing "adjust the car's air conditioning to 24℃" and sends this command directly to the infotainment system. Upon receiving the control command, the infotainment system can activate the air conditioning control module to perform the temperature adjustment.

[0089] For example, the first device is smart glasses, the associated device is the vehicle's infotainment system, and the intermediate device is the user's mobile phone. The smart glasses and the vehicle's infotainment system establish an indirect communication connection through the mobile phone. When the user is away from the vehicle, they utter the voice interaction command "Unlock the driver's door, [user's name]". The first processor sends the target audio to the server, which then parses the semantic information to generate a control command representing "unlock the driver's door". Based on the binding relationship between the intermediate device and the associated device, the control command is first sent to the mobile phone. The mobile phone forwards the control command to the vehicle's infotainment system via a mobile network or vehicle network. After receiving the control command from the mobile phone, the vehicle's infotainment system can activate the door lock control module to perform the unlocking operation.

[0090] As can be seen, users only need to interact with the first device via voice to control the first device itself and / or associated devices, without manually switching between operating objects or touching various associated devices, greatly simplifying the operation process. The server directly sends control commands to the target device, reducing intermediate transmission links and ensuring the immediacy of command execution, providing users with a smooth, instant-execution experience. Furthermore, the server handles computationally intensive operations such as semantic parsing, freeing the first device from the burden of complex calculations and command forwarding, effectively reducing its own energy consumption and meeting the low-power requirements of wearable devices such as smart glasses, thus extending device battery life. More importantly, the coverage of direct and indirect communication connections breaks down spatial limitations in device control, allowing users to manage multiple devices through a unified voice interaction interface in different scenarios. This enriches the application scenarios of voice processing methods and enhances the convenience and flexibility of user interaction with smart devices.

[0091] In addition, in some embodiments, the first processor may acquire semantic information corresponding to the target audio; generate control instructions based on the semantic information corresponding to the target audio; execute the control instructions, or send control instructions to the associated device.

[0092] After obtaining the semantic information corresponding to the target audio, the first processor can accurately match the preset response logic based on the semantic information and generate corresponding control commands. Then, the first processor can execute the control commands to quickly respond to the user's voice interaction commands.

[0093] Specifically, when the first processor obtains the semantic information corresponding to the target audio, it can rely on a locally deployed lightweight semantic parsing algorithm. This algorithm can be optimized for the computing resources and low power consumption requirements of the first device (such as smart glasses) without relying on the computing power support of external servers. This avoids network transmission delays and reduces energy consumption caused by data uploading.

[0094] The first processor can perform local preprocessing on the target audio, such as filtering residual noise using a built-in miniaturized VAD algorithm, and then perform semantic parsing based on a locally stored library of commonly used instruction keywords and a basic grammar rule library. The first processor can extract semantic information from the target audio without complex deep learning models, simply through keyword matching and simple sentence structure analysis. For example, if the target audio is "adjust its own brightness to medium," the first processor identifies the operation type by recognizing the keyword "adjust brightness," extracts parameters using "medium," and quickly parses the semantic information of "adjust the brightness of the first device display to medium." Similarly, if the target audio is "play locally collected light music," the first processor recognizes keyword combinations such as "play," "local collection," and "light music," clarifying that the user's core need is to trigger the local audio playback function.

[0095] When the first processor generates control instructions based on semantic information, it can encapsulate the control instructions according to a preset format to ensure that the control instructions can be accurately identified and executed by the first device itself or associated devices.

[0096] In scenarios involving the execution of control commands, after the first processor generates the control command, it can directly invoke its associated hardware modules or software functions to complete the execution, without relying on external devices or networks. This results in lower response latency and makes it particularly suitable for scenarios with high requirements for the immediacy of interaction. For example, if the first device is smart glasses, and the target audio is processed and parsed to reveal the semantic information "adjust its own volume to 30%", the first processor generates the corresponding control command and directly calls the local audio control module to adjust the playback volume of the smart glasses to 30%. The entire process requires no network interaction or device linkage, and the command is executed instantly.

[0097] In scenarios where control commands are sent to associated devices, the first processor can send control commands to the target associated device through the corresponding communication link based on the target associated device type implied in the semantic information.

[0098] This local semantic parsing and command generation design is suitable for scenarios with no network or poor network signal. Even if the first device cannot connect to the server, the first processor can still independently parse and execute common voice commands, ensuring the availability of basic voice interaction functions. It also further reduces the first device's power consumption and network dependence. The lightweight local algorithm does not require excessive computing power, and local or short-distance command transmission avoids the energy consumption of long-distance network communication, meeting the core needs of wearable devices such as smart glasses. Furthermore, through flexible modes of directly or indirectly controlling associated devices, users only need to use the first device as a unified interaction entry point to manage multi-level associated devices, eliminating the need to remember the interaction methods of different devices, significantly improving the convenience and scenario adaptability of voice interaction.

[0099] Additionally, in some embodiments, the associated device includes at least one of the following: A second device; the second device has a direct communication connection with the first device; the first device includes the first processor; Third device: The third device has a direct communication connection with the second device.

[0100] In addition, in some embodiments, the associated device includes at least one of the following: a second device; the second device having a direct communication connection with the first device; the first device including a first processor; and a third device: the third device having a direct communication connection with the second device.

[0101] Specifically, the second device, as a core interconnected device directly linked to the first device, can achieve direct communication with the first device using mature communication protocols such as Bluetooth, Wi-Fi, and automotive-grade Ethernet. This ensures strong link stability and low transmission latency, allowing for point-to-point transmission of control commands without any intermediate devices. After performing local semantic parsing of the target audio, the first processor generates corresponding semantic information and sends the encapsulated control commands to the second device via the direct communication link. Upon receiving the control commands, the second device can directly activate its corresponding functional modules to execute operations, quickly responding to user needs.

[0102] The third device establishes an indirect linkage path through a direct communication connection with the second device. It can receive control commands sent by the first device without establishing a direct link with the first device, leveraging the relay function of the second device. The second device, acting as a relay node, not only receives and forwards control commands but also ensures the integrity and security of command transmission, effectively overcoming the physical distance limitation between the first and third devices. After performing local semantic parsing of the target audio, the first processor generates corresponding control commands and sends them to the second device via the direct communication link. Upon receiving the control commands, the second device, based on its direct communication connection with the third device, forwards the complete control commands to the third device. Upon receiving the control commands, the third device activates its corresponding functional modules to execute operations, meeting the user's remote control needs.

[0103] This categorized design of associated devices ensures both the real-time transmission and execution of control commands in near-field scenarios and expands its application to long-field scenarios. Furthermore, both types of associated devices use the first device as a unified voice interaction entry point. The first processor only needs to select whether to send control commands directly or forward them via the second device based on the type of associated device, without needing to adapt to the communication protocols of different associated devices, thus simplifying the control logic of the first device. For wearable devices such as smart glasses, only low-power, near-field communication with the second device is required, without the need to bear the long-distance communication load with the third device, effectively reducing their own energy consumption. This aligns with the low-power, lightweight characteristics of these devices and lays a clear device link foundation for the accurate transmission and execution of subsequent control commands.

[0104] In addition, in some embodiments, when the associated device is a second device, a control command is sent to the second device to cause the second device to execute the control command; When the associated device is a third device, a control command is sent to the second device so that the second device forwards the control command to the third device and the third device executes the control command.

[0105] Specifically, the second device, as a core interconnecting device directly linked to the first device, can establish a stable link with the first device via communication protocols such as Bluetooth, Wi-Fi, and automotive-grade Ethernet, enabling direct transmission of control commands without intermediate nodes. For example, the first device could be smart glasses, and the second device could be an in-vehicle infotainment system (the user wears the smart glasses in a car setting, and the two establish a direct communication connection via automotive-grade Wi-Fi). The user can utter the voice interaction command, "Student XX, please adjust the in-vehicle volume to 50%." The first processor performs local semantic analysis of the target audio, generating semantic information representing "adjust the in-vehicle volume to 50%," and sends the control command to the in-vehicle infotainment system (the second device) via the direct communication link. Upon receiving the control command, the in-vehicle infotainment system can execute it, that is, invoking the audio control module to adjust the volume parameters.

[0106] The third device establishes a direct communication connection with the second device, and indirectly links with the first device through the relay function of the second device. This eliminates the need for a direct link between the first and third devices to transmit control commands. For example, the first device might be smart glasses, the second device might be a user's mobile phone (both establish a direct communication connection via Bluetooth), and the third device might be the vehicle's infotainment system (establishing a direct communication connection with the phone via vehicle networking). When the user is outdoors, away from the vehicle, and utters the voice command "Student XX, start the car's air conditioning and set the temperature to 24℃," the first processor performs local semantic analysis on the target audio and generates a control command representing "start the car's air conditioning and adjust the temperature to 24℃." This control command is then sent to the mobile phone (the second device) via the Bluetooth direct communication link. Upon receiving the control command, the mobile phone, based on its direct communication connection with the vehicle's infotainment system via vehicle networking, forwards the control command to the vehicle's infotainment system (the third device). Upon receiving the control command, the vehicle's infotainment system can execute it, that is, activate the air conditioning control module to start the cooling or heating system and adjust the interior temperature to 24℃.

[0107] This control command sending and forwarding logic provides users with a unified and flexible multi-device control solution. Users do not need to switch interaction entry points based on the type of associated device; they can always use the first device as the core of voice interaction to directly control the second device and indirectly control the third device, significantly simplifying the operation process. Furthermore, the first device does not need to establish a direct communication link with the third device; it only needs to maintain low-power, short-range communication with the second device, effectively reducing its own communication load and energy consumption, aligning with the low-power, lightweight characteristics of wearable devices such as smart glasses. In addition, the direct transmission mode ensures instant response to control commands in short-range scenarios, while the relay transmission mode breaks down physical distance limitations, allowing users to accurately control associated devices in different spatial scenarios. This improves the convenience and reliability of voice interaction, expands the application scope of voice processing methods, and further optimizes the user experience.

[0108] In addition, in some embodiments, the second or third device is an in-vehicle infotainment system included in the vehicle; the control commands include at least one of the following: Control commands used to trigger the information collection components included in the vehicle to perform in-vehicle information collection tasks; Control commands used to trigger the audio playback components included in the vehicle to perform audio playback tasks; Control commands are used to trigger adjustable components of the vehicle to perform adjustment tasks; the adjustable components include at least one of windows, wipers, seats, doors, lighting components, display components, and air conditioning components.

[0109] Among them, when the control command is used to trigger the information collection components included in the vehicle to perform in-vehicle information collection tasks, the information collection components included in the vehicle can be flexibly configured according to actual collection needs.

[0110] For example, the information acquisition components included in a vehicle can be temperature sensors, and the in-vehicle information acquisition task corresponding to the temperature sensors can be real-time detection of temperature data in various areas of the vehicle. The information collection components included in the vehicle can be humidity sensors, and the in-vehicle information collection task corresponding to humidity sensors can be real-time detection of in-vehicle air humidity. The information collection components included in the vehicle can be air quality sensors, and the in-vehicle information collection tasks corresponding to the air quality sensors can be detecting harmful gas indicators such as PM2.5 concentration and formaldehyde content in the vehicle. The information collection components included in the vehicle can be liveness detectors, and the in-vehicle information collection tasks corresponding to the liveness detectors can be detecting whether there are live animals such as pets or people left in the vehicle. The information collection components included in the vehicle can be vehicle-mounted cameras. The in-vehicle information collection tasks corresponding to the vehicle-mounted cameras can be capturing images of whether the seat belts of the people in the vehicle are being worn, the status of the passengers, and the status of the pets. The information collection components included in the vehicle may be microphone arrays, and the in-vehicle information collection tasks corresponding to the microphone arrays may be collecting in-vehicle voice commands and ambient noise intensity. The information collection components included in the vehicle can be tire pressure sensors, and the in-vehicle information collection task corresponding to the tire pressure sensors can be to detect the real-time tire pressure and temperature of each tire of the vehicle. The information collection components included in the vehicle can be fuel level sensors, and the in-vehicle information collection tasks corresponding to the fuel level sensors can be collecting data on the remaining fuel level in the vehicle's fuel tank and fuel consumption.

[0111] The information collection components included in the vehicle can be vehicle speed sensors, and the in-vehicle information collection tasks corresponding to vehicle speed sensors can be real-time detection of vehicle speed, acceleration and deceleration status. The information acquisition components included in the vehicle can be steering angle sensors, and the in-vehicle information acquisition tasks corresponding to steering angle sensors can be detecting steering wheel steering angle, steering speed and steering direction. The information acquisition components included in the vehicle can be battery voltage sensors, and the in-vehicle information acquisition tasks corresponding to the battery voltage sensors can be real-time monitoring of changes in vehicle battery voltage and charging / discharging status. The information collection components included in the vehicle can be ultrasonic radar, and the in-vehicle information collection task corresponding to ultrasonic radar can be to detect the distance and direction of obstacles around the vehicle. The information collection components included in the vehicle can be millimeter-wave radar, and the in-vehicle information collection tasks corresponding to millimeter-wave radar can be detecting the driving distance, relative speed and lane position of the vehicle in front. The information collection components included in the vehicle can be rain sensors, and the in-vehicle information collection tasks corresponding to the rain sensors can be detecting the amount and intensity of rainfall on the surface of the vehicle window glass. The information acquisition components included in the vehicle can be light sensors, and the in-vehicle information acquisition tasks corresponding to the light sensors can be detecting the light intensity and changes in light in the external environment. The information collection components included in the vehicle can be infrared sensors. The in-vehicle information collection tasks corresponding to infrared sensors can be detecting the body surface temperature of the people inside the vehicle and the ambient temperature outside the vehicle.

[0112] When the control command is a control command that triggers the audio playback components included in the vehicle to execute an audio playback task, the audio playback components included in the vehicle can be in-vehicle speakers, independent audio systems, zone audio modules, etc., and the corresponding audio playback tasks can be playing music, audiobooks, in-vehicle radio, voice interaction response feedback, movie audio, traffic information broadcasts, weather warning audio, etc.

[0113] When the control command is a control command that triggers the adjustable components of the vehicle to perform adjustment tasks, the adjustable components can accurately adapt to the operation scenario according to the user's interaction needs, and the corresponding adjustment tasks can cover diverse functional requirements.

[0114] For example, the adjustable component can be a car window, and the corresponding adjustment tasks can be opening, closing, adjusting the height (such as opening to one-third position, half-open state, fully open), one-button lifting control, smooth lifting in anti-pinch mode, and opening and closing the window locking function. Adjustable components can be windshield wipers, and the corresponding adjustment tasks can be turning them on and off, adjusting the wiping frequency (low-speed wiping, medium-speed wiping, high-speed wiping, automatic sensing wiping), adjusting the interval between intermittent wiping, triggering the water spray cleaning function (synchronous wiping and water spraying), and independent control of the rear windshield wipers. Adjustable components can be seats, and the corresponding adjustment tasks can be fore-and-aft sliding adjustment, height adjustment, backrest angle tilt adjustment, headrest height and fore-and-aft distance adjustment, lumbar support height and intensity adjustment, leg rest extension adjustment, seat rotation adjustment (for rear independent seats), seat heating function on / off and level switching (low, medium and high levels), seat ventilation function on / off and fan speed adjustment, and seat massage function on / off and mode switching (kneading, tapping, and circulation modes). Adjustable components can be car doors, and the corresponding adjustment functions can be unlocking, locking, electric soft closing, child safety lock opening and closing, electric opening / closing of sliding doors, door opening angle limit adjustment (to avoid scratches), and activation of electric door anti-pinch function. Adjustable components can be lighting components, and the corresponding adjustment tasks can be: turning low beam headlights on / off and adjusting brightness, turning high beam headlights on / off and automatically switching, turning turn signals on / off and adjusting flashing frequency, turning side marker lights on / off, turning fog lights (front fog lights, rear fog lights) on / off, adjusting brake light brightness, turning hazard warning lights on / off, turning ambient lights on / off, color temperature switching (warm light, cool light, multi-color gradient) and adjusting brightness, turning welcome lights on / off and switching projection patterns; Adjustable components can be display components, and the corresponding adjustment tasks can be turning the vehicle screen on / off, automatic and manual brightness adjustment, switching display interfaces (navigation interface, music interface, vehicle status monitoring interface, multimedia entertainment interface), split-screen display settings (simultaneously displaying navigation and music), screen rotation adjustment, font size and display resolution adjustment, screen sleep time setting, turning the head-up display (HUD) on / off and projection height, and brightness adjustment. Adjustable components can be air conditioning components, and the corresponding adjustment tasks can be turning the air conditioning system on / off, switching between cooling / heating modes, adjusting the target temperature, adjusting the fan speed (automatic fan speed, low speed, medium speed, high speed), switching the air outlet mode (face blowing mode, foot blowing mode, front windshield blowing mode, rear windshield blowing mode, multi-mode combination), adjusting the direction of the air outlet and the distribution of air volume, turning the air purification function on / off, turning the PM2.5 filtration function on / off, turning the dehumidification function on / off, and coordinating with seat heating / ventilation for temperature adjustment.

[0115] As can be seen, the embodiments of this application achieve deep adaptation of voice control to the in-vehicle scenario. Users can trigger a variety of functions of the vehicle device, such as information collection, audio playback, and component adjustment, simply by inputting voice interaction commands to the first device (such as smart glasses). There is no need to manually operate the vehicle screen or physical buttons, which greatly reduces distraction during driving and effectively improves driving safety.

[0116] Furthermore, the information collection components cover all dimensions such as vehicle operation, environmental monitoring, and personnel status, providing data support for the precise execution of control commands; the audio playback task balances functionality and entertainment, meeting diverse auditory needs; the adjustable components are refined to various scenario-based operations, adapting to different users' physical characteristics, usage habits, and driving scenarios, improving the comfort and personalized adaptability of the in-vehicle environment.

[0117] Furthermore, relying on the unified voice interaction interface of the first device, multi-dimensional control of in-vehicle devices can be achieved without switching the operation object, further enhancing the practicality and convenience of voice processing methods in in-vehicle scenarios and optimizing the user's in-vehicle interaction experience.

[0118] In some embodiments, the first processor plays response audio corresponding to the voice interaction command.

[0119] If the target device is the first device, the first processor can obtain the local execution information of the control command in real time without waiting for cross-device transmission delay, and the response audio can be played according to the local execution information.

[0120] For example, the first device is smart glasses, and the voice interaction command is "How is the weather today, classmate xx?" The corresponding control command is to search for and play "Today's weather information".

[0121] If the first device successfully retrieves today's weather information, it can play the audio message: "Today's weather information has been successfully retrieved. We are now reporting that today's weather is sunny and the temperature is 25 degrees Celsius."

[0122] If the first device fails to find today's weather information due to network interruption, lack of function support, or other reasons, the response audio can synchronously provide feedback on the reason for the failure and simple guidance, such as playing "Unable to obtain today's weather information temporarily. It may be that the current network is not connected. Please check the network status and try again," to help users quickly understand the problem.

[0123] If the target device is not the first device, the first processor can directly or indirectly (through the server) receive the target device's execution information of the control command, and play the response audio according to the received execution information.

[0124] For example, the first device is smart glasses, the target device is a car infotainment system, and the voice interaction command is "XX, open the car window." The corresponding control command is to trigger the car infotainment system to open the window. When the server forwards the execution information "The window has been successfully opened" to the first device, or when the first device directly receives the execution information "The window has been successfully opened" from the car infotainment system, it can play the response audio "The window has been successfully opened." If the car infotainment system does not respond or has a connection problem, it can directly or indirectly (through the server) provide feedback to the first device on the execution information of the control command (including task execution failure and the reason for failure). After receiving the execution information, the first device can play "Unable to open the window, please check the connection status between the car infotainment system and the smart glasses."

[0125] As can be seen, in scenarios where the target device is the primary device, the primary processor acquires local execution information in real time and provides rapid feedback. This avoids delays caused by cross-device transmission, allowing users to receive task results instantly. Furthermore, it simultaneously informs users of the reasons and provides guidance when a task fails, reducing user confusion caused by unknown faults and improving the efficiency and usability of the interaction. In cases where the target device is not the primary device, the primary processor receives execution information directly or via a server, ensuring that the status of cross-device interaction is clearly transmitted. Users do not need to additionally confirm the execution status of the linked devices; they can accurately know the task progress through audio responses. This simplifies the multi-device interaction process and avoids duplicate instructions or operational errors caused by information opacity.

[0126] Furthermore, regardless of whether the target device is the primary device, the response audio provides targeted feedback on the execution information of the control command. When successful, it clearly informs the execution status, and when unsuccessful, it provides actionable solutions. This forms a complete interactive loop of command input, response task execution, and execution information feedback, giving users a stronger sense of control over voice interaction and further improving the fluency, clarity, and user satisfaction of voice interaction, making it suitable for various usage scenarios involving multiple devices.

[0127] In addition, in some embodiments, the end point of the wake-up audio time acquisition is the start point of the control audio time acquisition; the duration of the wake-up audio is the target duration.

[0128] Specifically, the second processor determines the wake-up audio in two scenarios. Both scenarios rely on the low-power continuous acquisition characteristics of the second processor to achieve seamless connection between wake-up audio and control audio through time backtracking.

[0129] In the first scenario, the second processor detects that it has started collecting control audio. At this point, the second processor uses the moment it starts collecting control audio as a time reference, and traces back 1000 milliseconds to capture audio data within that target duration, identifying the captured audio data as the wake-up audio. For example, when the user continuously outputs "XX student is checking the weather," the second processor, at the instant it begins collecting the control audio "check the weather," will capture "XX student" 1000 milliseconds back, which will be the wake-up audio, avoiding content loss due to gaps in the collection process.

[0130] The second scenario involves the second processor detecting that the first processor has begun acquiring control audio. In this case, after activating its own control audio acquisition function, the first processor sends a acquisition start signal to the second processor. This signal informs the second processor that it has begun acquiring control audio. The second processor can use the moment it receives this acquisition start signal as a time reference and use the audio data captured within a target time period to serve as the wake-up audio. For example, when a user quickly says, "Student XX, turn on the air conditioner," the first processor is awakened, initiates control audio acquisition, and immediately sends an acquisition start signal to the second processor. The audio data that the second processor retrieves after receiving the acquisition start signal will precisely contain the complete wake-up audio of "Student XX," ensuring that no wake-up content is missed and accurately aligning with the acquisition timing of the first processor.

[0131] It should be noted that the target duration in this embodiment can be flexibly adjusted according to the length of common wake-up keywords and differences in user speech rate, to ensure that the recalled wake-up audio can fully cover the wake-up keywords without containing too many irrelevant segments, thereby reducing redundant data in subsequent processing.

[0132] This scenario-specific time reference determination method is compatible with both the second processor forwarding control audio and the first processor directly acquiring control audio, and it also ensures timing synchronization during cross-processor acquisition through a signal feedback mechanism. This avoids the timing gap problem in the wake-up detection to control acquisition process in traditional acquisition methods. Without increasing hardware costs and power consumption, it further improves the integrity of voice interaction commands and the adaptability of the solution, providing coherent and efficient basic data for subsequent audio processing and semantic parsing.

[0133] In some embodiments, both the first processor and the second processor are located in the first device, and the power consumption of the second processor is less than that of the first processor.

[0134] Specifically, the second processor can serve as a low-power monitoring unit in the first device, possessing the characteristic of continuous operation without significantly consuming device power. The core task of the second processor is to capture external voice signals in real time and analyze whether the external voice signals include wake-up audio based on preset wake-up audio features (such as wake-up word voiceprints and frequency features). Since it does not need to undertake complex data processing tasks, the second processor can remain active for extended periods, ensuring an immediate response to user wake-up operations.

[0135] The first processor, as the core unit responsible for the main data processing tasks, remains in a non-wake-up state when no wake-up signal is received, consuming far less energy than when the first processor is in a wake-up state. When the second processor detects a wake-up audio, it can generate a wake-up signal and send it to the first processor, triggering the first processor to switch from a non-wake-up state to a wake-up state. Subsequently, the first processor performs high-intensity processing tasks such as receiving control audio, audio splicing, voice separation, interacting with the server to obtain semantic information, and executing response tasks. After completing the tasks, it can return to a non-wake-up state, forming an energy-saving logic of wake-up on demand.

[0136] Correspondingly, in the case where the first device does not include the second processor, for example, the electronic device is smart glasses, the first processor is the processor included in the smart glasses; the second processor is the processor of the vehicle's infotainment system connected to the smart glasses. Since the vehicle's infotainment system can obtain continuous power, the second processor can maintain the state of detecting wake-up audio for a long time without consuming the battery life of the electronic device.

[0137] Furthermore, after detecting the wake-up audio, the second processor can send a wake-up signal to the first processor of the smart glasses through the communication link established with the smart glasses (such as Bluetooth Low Energy protocol).

[0138] The first processor can remain in a non-wake-up state before receiving a wake-up signal to minimize its own power consumption. Especially in driving scenarios, this design, in which the vehicle-mounted device is responsible for monitoring power consumption, can significantly reduce the power consumption of smart glasses and extend the duration of a single use.

[0139] Furthermore, after the first processor is awakened, it continues to perform tasks such as receiving control audio, audio splicing, and voice separation. It can also coordinate with the vehicle's functional modules as needed (such as interacting with the server through the vehicle's communication module to obtain semantic information, or calling the vehicle's audio system to perform voice broadcasts). After the task is completed, the smart glasses' first processor can switch back to the non-wake-up state, while the vehicle's second processor continues to monitor and wait for the next wake-up command, forming a highly efficient collaborative closed loop across devices.

[0140] As can be seen, in the embodiments of this application, when the first device includes a second processor, the division of labor and cooperation between the first processor and the second processor can realize the low-power continuous monitoring of the first processor and the high computing power of the second processor on demand, thereby reducing the overall power consumption of the first device and extending the battery life of the first device.

[0141] Without the second processor, the first device can further reduce its power consumption and simplify its hardware structure through cross-device collaboration between the first and second processors.

[0142] In addition, in some embodiments, the first processor may, in response to entering a wake-up state, control a first number of audio acquisition components among a plurality of audio acquisition components to be in an operating state; the first number is greater than a second number; the second number is the number of audio acquisition components in an operating state among a plurality of audio acquisition components when the first processor is in a non-wake-up state.

[0143] Specifically, where the first device includes a first processor and a second processor, the first device may also include multiple audio acquisition components (e.g., microphones). The audio acquisition components are used to acquire audio from the environment, such as wake-up audio and control audio.

[0144] When the first processor is in a non-wake-up state, in order to minimize the overall power consumption of the first device, only a small number (the second number) of audio acquisition components need to be kept running. These running audio acquisition components can work with the second processor to complete real-time monitoring of the wake-up audio.

[0145] Once the first processor enters the wake-up state in response to the received wake-up signal, it needs to accurately capture the control audio subsequently output by the user. To improve the acquisition quality of the control audio and enhance anti-interference capabilities, the first processor can control more audio acquisition components to start operating, bringing the number of operating audio acquisition components to a first number (the first number is greater than the second number). When multiple audio acquisition components work together, they can focus on the user's direction through beamforming technology, amplifying the user's voice signal; they can also filter environmental noise (such as background conversations and equipment operating noise) through multi-channel data comparison, and even compensate for signal distortion or loss that may occur in a single component, thus making the acquired control audio clearer and more complete.

[0146] In addition, in some embodiments, the first processor switches from a first operating frequency to a second operating frequency in response to an input wake-up signal; the second operating frequency is greater than the first operating frequency.

[0147] The first operating frequency is at a lower level compared to the second operating frequency because the first processor does not need to process complex tasks in the non-wake-up state. It only needs to maintain a basic standby or low-load operation state. For example, it does not need to receive control audio in voice interaction commands or perform response tasks (such as controlling device functions, interacting with servers, etc.). Therefore, the low operating frequency is sufficient to meet its needs in the non-wake-up state, while minimizing energy consumption.

[0148] The first processor, in the wake-up state, needs to undertake more computationally demanding tasks, such as receiving control audio from voice interaction commands, processing audio data (such as subsequent possible splicing or voice separation operations), sending semantic acquisition commands carrying target audio to the server, receiving semantic information fed back by the server, and finally executing the response tasks corresponding to the voice interaction commands (such as controlling electronic devices to turn on Bluetooth, play music, check the weather, etc.). These tasks all require more efficient computing power. Therefore, after receiving the wake-up signal, the first processor can respond to the wake-up signal, enter the wake-up state, and switch from the first operating frequency to the second operating frequency.

[0149] Specifically, the first processor can integrate a frequency control module. When this module detects a wake-up signal from the second processor, it can initiate a frequency adjustment process according to a preset frequency switching strategy. For example, if the first processor's first operating frequency is 1GHz and its second operating frequency is 2.4GHz, when the user speaks a wake-up keyword (such as "XX classmate"), the second processor detects the wake-up keyword and sends a wake-up signal to the first processor. Upon receiving the wake-up signal, the first processor's frequency control module will instantly increase its operating frequency from 1GHz to 2.4GHz. If the user then immediately speaks a control command (such as "Check today's weather"), the first processor, operating at a higher frequency, can quickly receive and process this control audio, ensuring the smooth execution of subsequent response tasks.

[0150] It is evident that by operating at a lower initial operating frequency in the non-wake-up state, the first processor can significantly reduce its own power consumption, thereby reducing the overall power consumption of the device it is connected to. For wearable electronic devices such as smartwatches and smart glasses, which have limited battery capacity and are sensitive to power consumption, this low operating frequency standby mode can effectively extend the battery life of the first device, which is beneficial to the user experience. Furthermore, upon receiving a wake-up signal, the first processor promptly switches to a higher second operating frequency, which provides more sufficient computing power to ensure that it can efficiently and quickly process various complex tasks in the wake-up state. This avoids problems such as task lag and response delays caused by insufficient computing power, allowing users to obtain a good user experience with instant response.

[0151] Figure 3 This is a flowchart illustrating a second speech processing method provided in an embodiment of this application. The second speech processing method provided in this application can be applied to a second processor. That is, the execution entity of the second speech processing method provided in this application can be a second processor.

[0152] The second processor can be a processor included in the connected device of the first device (including the device with the first processor). Alternatively, the second processor can be a processor included in the first device. The first device or its connected device can be an electronic device such as a mobile phone, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), handheld computer, netbook, personal digital assistant (PDA), wearable electronic device, virtual reality device, or in-vehicle infotainment system. Furthermore, wearable devices can be devices such as smart glasses, smartwatches, and smart bracelets; this application does not impose any limitations on these.

[0153] like Figure 3 As shown, the second speech processing method provided in this application embodiment may include the following steps: In response to detecting a voice interaction command including wake-up audio, the second processor sends a wake-up signal to the first processor, causing the first processor to enter a wake-up state in response to the input wake-up signal, and causing the first processor to receive control audio included in the voice interaction command while in the wake-up state.

[0154] The description of the second processor sending a wake-up signal to the first processor to put the first processor into a wake-up state, and the first processor receiving control audio including voice interaction commands when it is in the wake-up state, can be found in the description in the foregoing embodiments, and will not be repeated here.

[0155] Figure 4 This is a flowchart illustrating a third speech processing method provided in an embodiment of this application. The third speech processing method provided in this application can be applied to a server. That is, the executing entity of the third speech processing method provided in this application can be a server. The server can establish a communication connection with the first device.

[0156] like Figure 4 As shown, the third speech processing method provided in this application embodiment may include the following steps: S401: The server receives the target audio sent by the first processor.

[0157] Specifically, the server, as the core processing end of semantic parsing, can receive the target audio sent by the first processor through a preset communication interface (such as HTTP / HTTPS interface).

[0158] S402: The server performs editing on the target audio to obtain a valid audio segment.

[0159] The server can initiate an editing process on the target audio. The core purpose of editing is to remove invalid audio segments (such as silent audio segments, noisy audio segments, etc.) from the target audio in order to extract valid audio segments and reduce the computational consumption of subsequent semantic parsing.

[0160] Specifically, the server can invoke the VAD algorithm to analyze the target audio frame by frame: by detecting the energy value, frequency distribution, and spectral characteristics of each frame, it determines whether the frame is a valid audio segment. For example, if the energy value of an audio frame is below a preset threshold and there are no obvious human voice frequency characteristics, it is determined to be an invalid audio segment; otherwise, it is a valid audio segment including speech. Subsequently, the server can splice and integrate consecutive valid audio segments, while removing invalid audio segments at the beginning, end, and in the middle, forming a coherent and compact valid audio segment.

[0161] For example, in the target audio sent by the first processor, there are many invalid audio segments due to pauses in the user's speech, brief silences in the environment, and brief noises in the environment (such as invalid audio segments within the first 2 seconds, the middle 1.5 seconds, and the last 3 seconds). After the server accurately identifies these segments using the VAD algorithm, it can completely remove them, obtaining only valid audio segments that include the user's continuous speech. The audio duration will be compressed, effectively reducing the amount of data to be parsed subsequently.

[0162] S403: The server parses the semantic information corresponding to the target audio based on the valid audio segments.

[0163] Specifically, after acquiring a valid audio segment, the server can invoke a built-in semantic recognition algorithm (such as a deep learning-based integrated speech and semantic model) to perform deep analysis. This algorithm can comprehensively analyze the speech features (such as voiceprints and intonation), word combinations, grammatical structures, and semantic relationships in the valid audio segment, converting continuous speech signals into semantic information that clearly reflects the user's core needs. For example, if the valid audio segment is "Student xx, navigate to the city center hospital," the algorithm can parse out the semantic information of "query the location of the city center hospital and plan a navigation route." If there are polysemous words or ambiguous expressions, the server can also combine the user's historical interaction habits associated with the electronic device identifier (such as frequently used navigation preferences and past query records) to perform intent calibration, ensuring the accuracy of the semantic information.

[0164] S404: The server sends control commands to the target device based on the semantic information corresponding to the target audio.

[0165] The description of the server sending control commands to the target device based on the semantic information corresponding to the target audio can be found in the description in the foregoing embodiments, and will not be repeated here.

[0166] This application also provides a processor, including: a memory and a controller; the memory and the controller are coupled; wherein, the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the controller, the processor, as a first processor, performs the steps in the first speech processing method provided in this application.

[0167] This application also provides a processor, including: a memory and a controller; the memory and the controller are coupled; wherein, the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the controller, the processor acts as a second processor to execute the steps in the second speech processing method provided in this application.

[0168] Figure 5 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Figure 5 As shown, the server provided in this application embodiment includes: The receiving module is used to receive the target audio sent by the first processor; The editing module is used to edit the target audio to obtain valid audio segments; The semantic extraction module is used to parse the semantic information corresponding to the target audio based on valid audio segments; The sending module is used to send control commands to the target device based on the semantic information corresponding to the target audio.

[0169] This application also provides an electronic device, including the first processor provided in the above-described embodiments of this application, and / or including the second processor provided in the above-described embodiments of this application.

[0170] Among them, see Figure 6The first device provided in this application embodiment can establish a communication connection with a server to parse the target audio upon acquisition, thereby obtaining the semantic information corresponding to the target audio. Furthermore, the server can determine the target device based on this semantic information, generate control commands adapted to the target device, and send them to the target device. The first device does not require local deployment of semantic parsing algorithms, device matching logic, and multi-scenario control command generation modules; it only needs to undertake the basic functions of voice acquisition, wake-up response, and receiving execution feedback. Its hardware structure is significantly simplified, effectively reducing R&D and manufacturing costs. In addition, computationally intensive operations such as semantic parsing and command adaptation are all handled by the server, avoiding the high energy consumption caused by running complex algorithms locally on the first device, especially suitable for the low-power requirements of wearable devices such as smart glasses. Relying on the server's professional parsing capabilities and multi-device adaptation capabilities, the first device can accurately connect to different types of target devices (such as in-vehicle systems and mobile phones) through the server, achieving cross-device and cross-scenario voice interaction responses, while ensuring the accuracy and execution efficiency of control commands. This approach balances hardware costs and energy consumption with the smoothness, reliability, and scenario adaptability of voice interaction.

[0171] See Figure 7 This application also provides a vehicle. This vehicle is used to establish a communication connection with the electronic device provided in the embodiments of this application to achieve data interaction.

[0172] In some solutions, multiple embodiments of this application can be combined, and the combined solution can be implemented. Optionally, some operations in the processes of each method embodiment may be combined, and / or the order of some operations may be changed. Furthermore, the execution order between the steps of each process is merely exemplary and does not constitute a limitation on the execution order between steps; other execution orders are also possible. It is not intended to indicate that the execution order is the only possible order in which these operations can be performed. Those skilled in the art will conceive of various ways to reorder the operations described herein. In addition, it should be noted that the process details involved in one embodiment of this document are similarly applicable to other embodiments, or different embodiments may be combined.

[0173] Furthermore, some steps in the method embodiments can be equivalently replaced with other possible steps. Alternatively, some steps in the method embodiments may be optional and can be deleted in certain use cases. Or, other possible steps may be added to the method embodiments. Moreover, the various method embodiments can be implemented individually or in combination.

[0174] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the processor (first processor and second processor) can be divided into different functional modules to complete all or part of the functions described above.

[0175] In the several embodiments provided in this application, it should be understood that the disclosed methods, processors, servers, and electronic devices can be implemented in other ways. For example, the server embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into a system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or processors, and may be electrical, mechanical, or other forms.

[0176] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0177] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0178] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A speech processing method, characterized in that, Applied to a first processor, the method includes: When in a non-wake-up state, it enters a wake-up state in response to an input wake-up signal; the wake-up signal is emitted by the second processor upon detecting a voice interaction command including a wake-up audio. In response to entering the wake-up state, the system receives the control audio included in the voice interaction command.

2. The method according to claim 1, characterized in that, The response to entering the wake-up state, receiving the control audio included in the voice interaction command, includes: In response to entering the wake-up state, directly receive the control audio included in the voice interaction command; and / or, In response to entering the wake-up state, the system receives spliced ​​audio sent by the second processor; the spliced ​​audio is obtained by the second processor performing splicing processing on the wake-up audio and the control audio.

3. The method according to claim 2, characterized in that, Also includes: When the voice interaction command includes control audio, the wake-up audio and the control audio are spliced ​​together to obtain spliced ​​audio.

4. The method according to claim 2 or 3, characterized in that, Also includes: Perform a voice separation operation on the spliced ​​audio to obtain the target audio.

5. The method according to claim 4, characterized in that, Also includes: The target audio is sent to the server so that the server can parse the semantic information corresponding to the target audio, and the server can send control commands to the target device based on the semantic information corresponding to the target audio.

6. The method according to claim 5, characterized in that, The target device includes at least one of the following: A first device; the first device includes the first processor; Associated device; the associated device has a direct or indirect communication connection with the first device.

7. The method according to claim 4, characterized in that, Also includes: Obtain the semantic information corresponding to the target audio; Based on the semantic information corresponding to the target audio, control commands are generated; Execute the control command, or send the control command to the associated device.

8. The method according to claim 7, characterized in that, The associated device includes at least one of the following: A second device; the second device has a direct communication connection with the first device; the first device includes the first processor; Third device: The third device has a direct communication connection with the second device.

9. The method according to claim 8, characterized in that, Sending the control command to the associated device includes: If the associated device is the second device, the control command is sent to the second device to cause the second device to execute the control command; If the associated device is the third device, the control command is sent to the second device to cause the second device to forward the control command to the third device, and to cause the third device to execute the control command.

10. The method according to claim 9, characterized in that, The second device or the third device is an in-vehicle infotainment system included in the vehicle; the control command includes at least one of the following: Control commands used to trigger the information collection components included in the vehicle to perform in-vehicle information collection tasks; Control commands used to trigger the audio playback components included in the vehicle to perform audio playback tasks; Control commands are used to trigger adjustable components of a vehicle to perform adjustment tasks; the adjustable components include at least one of windows, windshield wipers, seats, doors, lighting components, display components, and air conditioning components.

11. The method according to any one of claims 1-10, characterized in that, Also includes: Play the response audio corresponding to the voice interaction command.

12. The method according to any one of claims 1-11, characterized in that, The end point of the wake-up audio time acquisition is the start point of the control audio time acquisition; the duration of the wake-up audio is the target duration.

13. The method according to any one of claims 1-12, characterized in that, Both the first processor and the second processor are located in the first device, and the power consumption of the second processor is less than that of the first processor.

14. The method according to claim 13, characterized in that, The response to entering the wake-up state, receiving the control audio included in the voice interaction command, further includes: In response to entering the wake-up state, a first number of audio acquisition components among the plurality of audio acquisition components are controlled to be in the running state; the first number is greater than a second number; the second number is the number of audio acquisition components in the plurality of audio acquisition components that are in the running state when the first processor is in the non-wake-up state.

15. The method according to any one of claims 1-14, characterized in that, The step of entering a wake-up state in response to an input wake-up signal further includes: In response to the input wake-up signal, the operating frequency is switched from a first operating frequency to a second operating frequency; the second operating frequency is greater than the first operating frequency.

16. A speech processing method, characterized in that, Applied to a second processor, the method includes: In response to detecting a voice interaction command including wake-up audio, a wake-up signal is sent to a first processor to cause the first processor to enter a wake-up state in response to the input wake-up signal, and to cause the first processor to receive control audio included in the voice interaction command when entering the wake-up state.

17. A processor, characterized in that, include: A memory and a controller; the memory and the controller are coupled; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the controller, cause the processor to perform the steps of the speech processing method according to any one of claims 1-15.

18. A processor, characterized in that, include: A memory and a controller; the memory and the controller are coupled; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the controller, cause the processor to perform the steps of the speech processing method of claim 16.

19. An electronic device, characterized in that, Includes the processor of claim 17, and / or includes the processor of claim 18.

Images