Audio system, wearable audio device and control method

Abstract

Provided in the present application are an audio system, a wearable audio device and a control method. The audio system is used for: collecting audio signals; identifying a control instruction corresponding to an audio signal of a target sound among the audio signals, wherein the target sound is a sound produced by the mechanical contact of a body part of a user; and executing a control operation corresponding to the control instruction. The audio system generally has a sound pickup function and requires no special hardware. In this way, a software algorithm for identifying a control instruction on the basis of a target sound is easy to port. In addition, the target sound may be a sound produced by tapping on bones, chattering teeth, rubbing the skin, etc. The user may define, on the basis of requirements, the manner in which the mechanical contact of a body part of the user produces a sound for triggering the control instruction, thereby achieving a high degree of customizability. Moreover, besides manual operation, the user may also perform operations by other means, such as clenching the teeth. Accordingly, the present application is applicable to various application scenarios, such as during exercise or when both hands are occupied.

Description

Audio systems, wearable audio devices and control methods

[0001] This application claims priority to Chinese Patent Application No. 202510005617.0, filed on January 2, 2025, entitled "Audio System, Wearable Audio Device and Control Method", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of media technology, and in particular to an audio system, wearable audio device and control method. Background Technology

[0003] With the maturation of multi-touch technology and advancements in technologies such as accelerometers and pressure sensors, headphone control methods have undergone significant changes. Traditional physical button controls are gradually being replaced by more advanced and convenient touch controls. This shift not only enhances the user experience but also brings more functionality and flexibility to headphones.

[0004] Currently, some headphones on the market allow users to play / pause audio, switch songs, and answer calls by tapping or pressing the earphone stem. This operation method is intuitive and easy to use, allowing users to complete complex operations with simple gestures. In addition, some headphones support volume adjustment by sliding the earphone stem up and down; this sliding operation is convenient and allows users to control the volume more precisely. Some high-end headphones also support single, double, triple, or long presses to achieve different functions; for example, a single press plays or pauses music, a double press skips to the next song, a triple press skips to the previous song, and a long press activates the voice assistant or rejects a call. This multi-press design allows users to flexibly customize the headphone's functions according to their needs.

[0005] However, touch control solutions have many problems: for example, special hardware such as touch sensors and pressure sensors need to be set on the headphones, resulting in low portability of the touch control solution; for another example, the control commands corresponding to the same touch operation are basically the same on different headphones, resulting in low customization; for yet another example, users must use their hands to operate, which makes it difficult to operate in scenarios where they are in motion or their hands are occupied; and so on. Summary of the Invention

[0006] In view of this, this application provides an audio system, a wearable audio device, and a control method. The audio system requires no special hardware, the control method used in the audio system is highly portable, the control method of the audio system is highly customizable, and it has a wide range of application scenarios.

[0007] In a first aspect, this application provides an audio system for acquiring audio signals; subsequently, identifying a control command corresponding to an audio signal of a target sound in the audio signal, wherein the target sound is a sound generated by mechanical contact of a user's body part; and then executing a control operation corresponding to the control command.

[0008] Mechanical contact may include, but is not limited to, striking, collision, and friction. User body parts may include, but are not limited to, bones, teeth, and skin. Mechanical contact involving user body parts can include mechanical contact between other objects and user body parts, or mechanical contact between user body parts themselves.

[0009] In other words, this application controls the audio system by picking up the sounds generated by the mechanical contact of user body parts. Since most audio systems have built-in sound pickup capabilities, no special hardware is required. Therefore, the software algorithm for recognizing control commands based on the sounds generated by the mechanical contact of user body parts is easily portable and can be ported to most audio systems. Secondly, the sounds generated by the mechanical contact of user body parts are diverse, including sounds from striking bones, teeth colliding, and skin rubbing. This allows users to customize the way the sounds generated by the mechanical contact of user body parts are used to trigger control commands, offering a high degree of customization and meeting individual user needs. Thirdly, in addition to using hands, users can also operate the system using other methods such as biting their teeth. Therefore, this application is applicable to various scenarios, such as sports or when both hands are occupied, making its application scenarios rich.

[0010] Furthermore, there are many combinations of sounds generated by mechanical contact with user body parts. Therefore, compared with the prior art, this application has more ways to control the audio system and more flexible control operation of the audio system.

[0011] Furthermore, compared to traditional methods of controlling audio systems, this application controls the audio system through sound generated by mechanical contact of the user's body parts, which is less likely to attract attention, offers greater privacy, and is suitable for certain special scenarios such as conferences.

[0012] For example, the audio signal acquired by the audio system may include an audio signal obtained by acquiring the sound generated by mechanical contact of a user's body parts via a bone conduction path, and / or an audio signal obtained by acquiring the sound generated by mechanical contact of a user's body parts via an air conduction path.

[0013] For example, control commands can control system functions (such as Bluetooth function, wireless function, positioning function, voice assistant function, etc.), control applications (such as audio applications, broadcast applications, audiobook applications, navigation applications, settings applications, etc.), audio volume control (such as increasing volume, decreasing volume), music switching, music playback / pause, making / answering / hanging up calls, etc., and this application does not limit this.

[0014] For example, the audio system of this application may include only wearable audio devices, or the audio system may include wearable audio devices and other terminal devices.

[0015] For example, wearable audio devices such as headphones, smartwatches, virtual reality (VR) headsets, augmented reality (AR) headsets, AR / VR glasses, smart bracelets, smart necklaces, and smart glasses.

[0016] For example, other terminal devices can refer to terminal devices other than wearable audio devices, such as mobile phones, tablets, laptops, personal computers, game consoles, in-vehicle terminals, smart TVs, smart speakers, etc.

[0017] According to the first aspect, identifying the control command corresponding to the target audio signal in the audio signal includes: detecting the audio sequence corresponding to the target audio signal; and identifying the control command based on the audio sequence.

[0018] The audio sequence can be composed of audio signals generated by repeated mechanical contact of the user's body parts. This combination method makes the audio sequence contain more information, that is, the audio sequence has diversity. The diversity of the audio sequence means that the corresponding control commands are also diverse, which allows the user to perform more diverse control on the audio system.

[0019] According to the first aspect, or any implementation of the first aspect above, based on the audio sequence, identifying the control instruction includes: based on the audio sequence, determining a target audio sequence from multiple preset audio sequences, where each preset audio sequence corresponds to one or more sets of control instructions, and the multiple sets of control instructions correspond to different scenarios; and determining the control instruction corresponding to the target audio sequence as the control instruction corresponding to the audio signal of the target sound.

[0020] This system allows for the pre-setting of preset audio sequences and corresponding control commands. During use, it can search for a preset audio sequence (i.e., the target audio sequence) that matches the audio sequence detected from the target sound's audio signal. Correspondingly, the control command for the target audio sequence is the control command for the target sound's audio signal. This allows for the rapid determination of the control command for the acquired target sound's audio signal.

[0021] For example, a set of control instructions may include one or more sub-control instructions (these sub-control instructions are logically related). In the case where a set of control instructions includes one sub-control instruction, this sub-control instruction may be such as pausing or playing audio, answering or hanging up a call, or skipping to the next track. In the case where a set of control instructions includes multiple sub-control instructions, these multiple sub-control instructions may be such as lowering the volume and stopping the volume from decreasing, or raising the volume and stopping the volume from increasing, or opening the camera and taking a picture.

[0022] For example, the same preset audio sequence can correspond to a set of control commands in different scenarios (or, a set of control commands in different scenarios can correspond to the same preset audio sequence). For instance, preset audio sequence 1 corresponds to a set of control commands for the audio playback scenario: pause audio; a set of control commands for the audio pause playback scenario: play audio; and a set of control commands for the incoming call scenario: answer the phone. Thus, when the phone receives a call, the target sound corresponding to preset audio sequence 1 can be emitted to answer the call; when music is playing on the phone, the target sound corresponding to preset audio sequence 1 can be emitted to pause music playback; and when music is paused on the phone, the target sound corresponding to preset audio sequence 1 can be emitted to play music.

[0023] For example, the preset audio sequences corresponding to a set of control commands can also be different in different scenarios. For instance, answering a phone call corresponds to preset audio sequence 1, and hanging up a phone call corresponds to preset audio sequence 2.

[0024] According to the first aspect, or any implementation of the first aspect above, the preset audio sequence is either user-defined or set by the audio system. This allows users to set preset audio sequences according to their own usage habits, making it easier for them to quickly learn how to control the audio system using a target sound.

[0025] According to the first aspect, or any implementation of the first aspect above, the control commands corresponding to the preset audio sequences are either user-defined or set by the audio system. This allows users to set preset audio sequences for frequently used control commands according to their own usage habits, facilitating quick control of the audio system later.

[0026] According to the first aspect, or any implementation of the first aspect above, the audio sequence is a combination of long and short audio signals of the same target sound.

[0027] Assuming the target sound is the sound produced by a user biting their teeth (hereinafter referred to as dental sounds), the audio sequence can be a combination of audio signals of long dental sounds and short dental sounds. For example, the audio sequence can consist of audio signals of two short dental sounds; or, for another example, the audio sequence can consist of audio signals of three short dental sounds; or, for yet another example, the audio sequence can consist of audio signals of two long dental sounds and one short dental sound.

[0028] It should be understood that long and short dentate sounds are relative. The type of dentate sound can be determined by the interval between the current dentate sound and the next dentate sound. A long dentate sound is defined as one whose interval with the next dentate sound is greater than a first threshold (e.g., 0.5 seconds); a short dentate sound is defined as one whose interval with the next dentate sound is less than a second threshold (e.g., 0.2 seconds).

[0029] According to the first aspect, or any implementation of the first aspect above, the audio sequence is a combination of audio signals of at least two target sounds.

[0030] For example, the audio sequence can be a combination of audio signals of any two or three of the target sounds, such as the sound of striking bones, the sound of teeth colliding, or the sound of rubbing skin.

[0031] According to the first aspect, or any implementation of the first aspect above, the target sound includes at least one of the following: a sound produced by striking bones, a sound produced by teeth colliding, or a sound produced by rubbing skin.

[0032] For example, the sound produced by teeth colliding can include the sound of a user biting their teeth.

[0033] For example, the sounds produced by tapping bones can include: the sounds produced by a user tapping facial bones, the sounds produced by a user tapping around the ear, the sounds produced by a user tapping the skull, or the sounds produced by a user tapping the wrist / back of the hand. Facial bones can include, for example, the cheekbone, frontal bone, and jawbone.

[0034] For example, the sound produced by rubbing the skin can include the sound produced by a user rubbing their ear.

[0035] Among them, compared with other methods of generating target sounds, generating target sounds by biting teeth does not require the use of hands, and is more private and convenient.

[0036] According to the first aspect, or any implementation of the first aspect above, the audio system is also used to output the execution result of the control operation. This allows the user to be informed whether the control of the audio system via the target sound was successful.

[0037] The execution result of the control operation can include success or failure.

[0038] One possible approach is to play audio recordings of the results of the control operation.

[0039] One possible approach is to display the results of the control operation.

[0040] One possible approach is to provide feedback on the execution result through vibration. For example, the result could be provided by the vibration of a wearable audio device (such as headphones). Another example is the vibration of other terminal devices (such as a mobile phone). The vibration pattern for successful execution can differ from that for failed execution; for instance, a successful execution might be characterized by a single long vibration, while a failed execution might be characterized by multiple short vibrations.

[0041] One possible approach is to provide feedback on the execution result in the form of a ringtone / sound effect. For example, the execution result could be provided by having a ringtone / sound effect played by a wearable audio device (such as headphones).

[0042] According to the first aspect, or any implementation of the first aspect above, the audio system is further configured to output confirmation information regarding whether the control operation is performed; and to perform the control operation upon receiving confirmation from the user. This reduces the risk of erroneous operations.

[0043] In one possible approach, a confirmation screen can be displayed indicating whether the control operation should be performed, the confirmation screen including a denial option and a confirmation option; the control operation is performed in response to the user's click on the confirmation option.

[0044] In one possible approach, a confirmation voice prompt can be played asking whether a control operation should be performed; in response to the user's confirmation voice prompt, the control operation is performed.

[0045] In one possible approach, confirmation of whether the control operation has been performed can be provided via vibration. For example, a wearable audio device (such as headphones) could vibrate to provide confirmation. Alternatively, other terminal devices (such as mobile phones) could vibrate to provide confirmation.

[0046] In one possible approach, confirmation of whether the control operation was performed could be provided in the form of a ringtone / sound effect. For example, a wearable audio device (such as headphones) could play a ringtone / sound effect to provide confirmation of whether the control operation was performed.

[0047] It should be understood that the vibration pattern for confirming whether the control operation has been performed may differ from the vibration pattern for confirming the execution result; and the sound effect / ringtone for confirming whether the control operation has been performed may differ from the sound effect / ringtone for confirming the execution result.

[0048] According to the first aspect, or any implementation of the first aspect above, the audio system is further configured to output cancellation information indicating whether the execution of the control operation has been cancelled; upon receiving a user's confirmation cancellation instruction, the execution of the control operation is cancelled. Thus, when a user needs to cancel a control operation on the audio system via a target sound, the control operation can also be cancelled.

[0049] In one possible approach, an undo interface for the control operation can be displayed, the undo interface including an undo option; in response to a user clicking the undo option, the execution of the control operation is undone.

[0050] In one possible approach, a voice prompt asking whether to cancel the control operation can be played; in response to the user's confirmation to cancel the control operation, the control operation can be canceled.

[0051] In one possible approach, feedback regarding whether the control operation has been revoked could be provided via vibration. For example, a wearable audio device (such as headphones) could vibrate to provide feedback on whether the control operation has been revoked. Alternatively, other terminal devices (such as mobile phones) could vibrate to provide feedback on whether the control operation has been revoked.

[0052] One possible approach is to provide feedback on whether the control operation has been undone via a ringtone / sound effect. For example, a wearable audio device (such as headphones) could play a ringtone / sound effect to provide feedback on whether the control operation has been undone.

[0053] It should be understood that the vibration pattern for feedback on whether to cancel the execution of the control operation is different from the vibration pattern for feedback on whether to confirm the execution of the control operation and the execution result; and the sound effect / ringtone for feedback on whether to cancel the execution of the control operation is different from the sound effect / ringtone for feedback on whether to confirm the execution of the control operation and the execution result.

[0054] According to the first aspect, or any implementation of the first aspect above, the audio system is further configured to output stop information indicating whether to stop executing the control operation; upon receiving a user's confirmation to stop execution, the control operation is stopped. In this way, the user can control the audio system at any time and stop the user's control operation on the audio system via a target sound.

[0055] In one possible approach, a stop execution interface for the control operation can be displayed, which includes a stop option; in response to a user clicking the stop option, the execution of the control operation is stopped.

[0056] In one possible approach, a voice prompt asking whether to stop the control operation can be played; in response to the user's confirmation to stop the control operation, the control operation is stopped.

[0057] In one possible approach, feedback on whether to stop the control operation could be provided via vibration. For example, a wearable audio device (such as headphones) could vibrate to provide feedback on whether to stop the control operation. Alternatively, other terminal devices (such as mobile phones) could vibrate to provide feedback on whether to stop the control operation.

[0058] In one possible approach, feedback on whether to stop the control operation could be provided in the form of a ringtone / sound effect. For example, a wearable audio device (such as headphones) could play a ringtone / sound effect to provide feedback on whether to stop the control operation.

[0059] It should be understood that the vibration pattern of the stop message indicating whether the control operation has been stopped may differ from the vibration pattern of the cancellation message indicating whether the control operation has been revoked, the vibration pattern of the confirmation message indicating whether the control operation has been executed, and the vibration pattern of the execution result. Similarly, the sound effect / ringtone of the stop message indicating whether the control operation has been stopped may differ from the sound effect / ringtone of the cancellation message indicating whether the control operation has been revoked, the sound effect / ringtone of the confirmation message indicating whether the control operation has been executed, and the sound effect / ringtone of the execution result.

[0060] According to the first aspect, or any implementation of the first aspect above, the audio system includes wearable audio devices and other terminal devices;

[0061] This wearable audio device is used to collect and transmit audio signals.

[0062] The other terminal device is used to identify the control command corresponding to the audio signal of the target sound in the audio signal; and to execute the control operation corresponding to the control command.

[0063] For example, wearable audio devices include headphones, while other terminal devices include mobile phones, tablets, etc.

[0064] According to the first aspect, or any implementation of the first aspect above, the audio system includes wearable audio devices and other terminal devices;

[0065] This wearable audio device is used to collect and transmit audio signals.

[0066] The other terminal device is used to identify the control command corresponding to the audio signal of the target sound in the audio signal and send the control command;

[0067] The wearable audio device is also used to execute the control operations corresponding to the control command.

[0068] For example, wearable audio devices include smart glasses, while other terminal devices include mobile phones, tablets, etc.

[0069] According to the first aspect, or any implementation of the first aspect above, the audio system includes wearable audio devices and other terminal devices;

[0070] This wearable audio device is used to collect and transmit audio signals.

[0071] The other terminal device is used to identify the control command corresponding to the audio signal of the target sound in the audio signal, the control command including a first control command and a second control command; execute the control operation corresponding to the first control command and send the second control command;

[0072] The wearable audio device is also used to execute control operations corresponding to the second control command.

[0073] For example, wearable audio devices are head-mounted displays, while other terminal devices include mobile phones, tablets, etc.

[0074] For example, the first control command is to pause, and the second control command is to open the display menu.

[0075] According to the first aspect, or any implementation of the first aspect above, the audio system includes wearable audio devices and other terminal devices;

[0076] This wearable audio device is used to collect audio signals; identify the control command corresponding to the audio signal of the target sound in the audio signal; and send the control command.

[0077] The other terminal device is used to execute the control operation corresponding to the control command.

[0078] For example, wearable audio devices are head-mounted displays, while other terminal devices include mobile phones, tablets, etc.

[0079] According to the first aspect, or any implementation of the first aspect above, the audio system includes wearable audio devices and other terminal devices;

[0080] The wearable audio device is used to collect audio signals; identify control commands corresponding to the audio signals of target sounds in the audio signals, the control commands including a first control command and a second control command; execute the control operation corresponding to the second control command and send the first control command;

[0081] The other terminal device is used to execute the control operation corresponding to the first control command.

[0082] For example, wearable audio devices are head-mounted displays, while other terminal devices include mobile phones, tablets, etc.

[0083] For example, the first control command is to pause, and the second control command is to open the display menu.

[0084] According to the first aspect, or any implementation of the first aspect above, the audio system includes a wearable audio device.

[0085] This wearable audio device is used to collect audio signals; identify the control commands corresponding to the audio signals of target sounds in the audio signals; and execute the control operations corresponding to the control commands.

[0086] For example, wearable audio devices include head-mounted displays, smartwatches, and so on.

[0087] Secondly, this application provides a wearable audio device for collecting audio signals; identifying control commands corresponding to audio signals of target sounds in the audio signals, wherein the target sounds are sounds generated by mechanical contact of a user's body parts; and sending the control commands to other terminal devices, wherein the control commands are used to instruct the other terminal devices to perform corresponding control operations.

[0088] It should be understood that the wearable audio device can also perform some of the steps performed by the aforementioned audio system, which will not be elaborated here.

[0089] Thirdly, this application provides a terminal device for receiving audio signals sent by a wearable audio device; identifying a control command corresponding to an audio signal of a target sound in the audio signal, wherein the target sound is a sound generated by mechanical contact of a user's body part; and executing a control operation corresponding to the control command.

[0090] It should be understood that this terminal device can also perform some of the steps executed by the aforementioned audio system, which will not be elaborated upon here. This terminal device corresponds to the other terminal devices in the aforementioned audio system.

[0091] Fourthly, this application provides a control method, which includes: acquiring an audio signal; identifying a control command corresponding to an audio signal of a target sound in the audio signal, wherein the target sound is a sound generated by mechanical contact of a user's body part; and executing a control operation corresponding to the control command.

[0092] It should be understood that the control method may also include the steps performed by the audio system described above, which will not be repeated here.

[0093] Fifthly, this application provides a command recognition method for wearable devices, the method comprising: acquiring an audio signal; identifying a control command corresponding to an audio signal of a target sound in the audio signal, wherein the target sound is a sound generated by mechanical contact of a user's body part; and sending the control command to other terminal devices, wherein the control command is used to instruct the other terminal devices to perform a corresponding control operation.

[0094] It should be understood that the instruction recognition method may also include the steps of the instruction recognition process performed by the audio system described above, which will not be elaborated here.

[0095] Sixthly, this application provides a control method for a terminal device, the method comprising: receiving an audio signal sent by a wearable audio device; identifying a control command corresponding to an audio signal of a target sound in the audio signal, wherein the target sound is a sound generated by mechanical contact of a user's body part; and executing a control operation corresponding to the control command.

[0096] It should be understood that the control method may also include the steps performed by the audio system described above, which will not be repeated here.

[0097] Seventhly, this application provides a control device, which includes: an audio acquisition module for acquiring audio signals; an instruction recognition module for recognizing a control instruction corresponding to an audio signal of a target sound in the audio signal, wherein the target sound is a sound generated by mechanical contact of a user's body part; and an instruction execution module for executing the control operation corresponding to the control instruction.

[0098] It should be understood that the control device can also be used to perform the steps performed by the audio system described above, which will not be elaborated here.

[0099] Eighthly, this application provides a chip including one or more interface circuits and one or more processors; the one or more processors receive or transmit data through the one or more interface circuits, and when the one or more processors execute computer instructions, cause the electronic device to perform the method in the fourth aspect or any possible implementation of the fourth aspect; or, cause the electronic device to perform the method in the fifth aspect or any possible implementation of the fifth aspect; or, cause the electronic device to perform the method in the sixth aspect or any possible implementation of the sixth aspect.

[0100] The electronic device can be a wearable audio device or other terminal device.

[0101] Ninthly, this application provides a computer-readable storage medium storing a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method of the fourth aspect or any possible implementation thereof; or, causes the computer or processor to perform the method of the fifth aspect or any possible implementation thereof; or, causes the computer or processor to perform the method of the sixth aspect or any possible implementation thereof.

[0102] In a tenth aspect, this application provides a computer program product comprising computer instructions that, when executed by a computer or processor, cause the computer or processor to perform the method of the fourth aspect or any possible implementation thereof; or, cause the computer or processor to perform the method of the fifth aspect or any possible implementation thereof; or, cause the computer or processor to perform the method of the sixth aspect or any possible implementation thereof.

[0103] In this embodiment, the computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above. Attached Figure Description

[0104] Figure 1A is a schematic diagram of an application scenario according to an embodiment of this application;

[0105] Figure 1B is a schematic diagram of another application scenario of the present application embodiment;

[0106] Figure 1C is a schematic diagram of another application scenario of the present application embodiment;

[0107] Figure 1D is a schematic diagram of another application scenario of the present application embodiment;

[0108] Figure 2 is a structural block diagram of an audio system 200 according to an embodiment of this application;

[0109] Figure 3 is a structural block diagram of another audio system 300 according to an embodiment of this application;

[0110] Figure 4 is a structural block diagram of another audio system 400 according to an embodiment of this application;

[0111] Figure 5 is a structural block diagram of another audio system 500 according to an embodiment of this application;

[0112] Figure 6 is a structural block diagram of another audio system 600 according to an embodiment of this application;

[0113] Figure 7 is a structural block diagram of another audio system 700 according to an embodiment of this application;

[0114] Figure 8A is a schematic diagram of a mobile phone interface according to an embodiment of this application;

[0115] Figure 8B is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0116] Figure 8C is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0117] Figure 8D is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0118] Figure 8E is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0119] Figure 9 is a schematic diagram of a control process 900 according to an embodiment of this application;

[0120] Figure 10A is a schematic diagram of the system framework of another audio system 1000 according to an embodiment of this application;

[0121] Figure 10B is a schematic diagram of another control process 1000 according to an embodiment of this application;

[0122] Figure 11 is a schematic diagram of the system framework of another audio system 1100 according to an embodiment of this application;

[0123] Figure 12A is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0124] Figure 12B is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0125] Figure 12C is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0126] Figure 12D is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0127] Figure 13 is a schematic diagram of another mobile phone interface according to an embodiment of this application;

[0128] Figure 14 is a schematic diagram of the structure of a device provided in an embodiment of this application. Detailed Implementation

[0129] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0130] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0131] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0132] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0133] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0134] In the embodiments of this application, the modules / components shown in the framework diagram (or structural diagram or system diagram) are merely examples of this application. The actual framework (or structure or system) may include more or fewer modules / components than those shown in the diagram, or may have different component configurations. Furthermore, the various components / modules shown in the diagrams may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0135] For example, the audio system of this application may include only wearable audio devices, or it may include wearable audio devices and other terminal devices. Wearable audio devices may include headphones, smartwatches, VR headsets, and smart glasses; furthermore, wearable audio devices may also include other devices, such as other head-mounted displays like augmented reality (AR) headsets, AR / VR glasses, smart bracelets, smart necklaces, etc., which are not limited in this application. Other terminal devices may refer to terminal devices other than wearable audio devices, such as mobile phones, tablets, laptops, personal computers, game consoles, in-vehicle terminals, smart TVs, smart speakers, etc. The following describes some application scenarios of the audio system of this application.

[0136] Figure 1A is a schematic diagram of an application scenario according to an embodiment of this application. Figure 1A shows some application scenarios of an audio system including headphones and a mobile phone, such as movie scenarios, music scenarios, call scenarios, and game scenarios.

[0137] Referring to Figure 1A, users can connect their mobile phones to headphones for communication. While watching movies, listening to music, making calls (including video and voice calls), or playing games, the mobile phone can send audio from the movie, music, the other party's voice, and game effects and background music to the headphones. The headphones then play these audio components. Using headphones ensures user privacy and enhances the listening experience.

[0138] Figure 1B is a schematic diagram of another application scenario of the present application embodiment. Figure 1B shows an application scenario that only includes the audio system of a smartwatch, such as a sports scenario.

[0139] Referring to Figure 1B, in a sports scenario, users can wear a smartwatch for health monitoring, such as heart rate, steps, and exercise speed. In addition, the smartwatch also has common functions such as timer, alarm clock, notification reminders, music playback, and call functionality, meeting users' daily needs. This way, users don't need to carry a phone during exercise, reducing fatigue from carrying heavy items and allowing them to focus more on their workout.

[0140] Figure 1C is a schematic diagram of another application scenario of the present application. Figure 1C shows some application scenarios of an audio system that only includes a virtual reality (VR) headset, such as VR movie scenes, VR concert scenes, VR game scenes, etc.

[0141] Referring to Figure 1C, users can wear VR headsets to watch movies, listen to concerts, play games, and more, enhancing their immersion and experience. When watching a movie, the VR headset's display panel shows the movie footage, and the headset's headphones play the movie's audio. When listening to a concert, the VR headset's display panel shows the musicians performing on stage, and the headset's headphones play the music. When playing games, the VR headset's display panel shows the game footage, and the headset's headphones play the game's sound effects and background music.

[0142] Figure 1D is a schematic diagram of another application scenario of an embodiment of this application. Figure 1D shows an application scenario of an audio system including smart glasses and a mobile phone, such as a navigation scenario.

[0143] Referring to Figure 1D, after wearing smart glasses, users can connect them to their mobile phones. When navigation is needed, users can launch a navigation app on their phones; afterwards, the phone can send navigation data (including navigation information and audio) to the smart glasses. The smart glasses can display navigation information (such as navigation routes) on the lenses and play navigation audio through the glasses' earpieces. This frees up the user's hands, allowing them to focus on the road and ensuring safety, without needing to look at their phone.

[0144] It should be understood that the audio system of this application may also include other application scenarios, and this application does not limit this. Furthermore, the mobile phones in Figures 1A and 1D can be replaced with other devices, and this application does not limit this.

[0145] It should be understood that the communication connection between other terminal devices and wearable audio devices can be a wired communication connection or a wireless communication connection.

[0146] Figure 2 is a structural block diagram of an audio system 200 according to an embodiment of this application. The audio system 200 in Figure 2 includes a wearable audio device 210 and other terminal devices 220.

[0147] Referring to Figure 2, the other terminal device 220 includes a first communication module 221, an instruction recognition module 222, and an instruction execution module 223; the wearable audio device 210 includes an audio acquisition module 212 and a second communication module 211.

[0148] For example, the audio acquisition module 212 can output the acquired audio signal 10 to the second communication module 211; then, the second communication module 211 outputs the audio signal 10 to the first communication module 221. After that, the first communication module 221 outputs the audio signal 10 to the instruction recognition module 222; then, the instruction recognition module 222 can recognize the control instruction 11 and output the control instruction 11 to the instruction execution module 223; then, the instruction execution module 223 executes the control operation corresponding to the control instruction 11.

[0149] For example, the wearable audio device 210 in Figure 2 can be headphones, and other terminal devices 220 can be mobile phones, tablets, etc.

[0150] Figure 3 is a structural block diagram of another audio system 300 according to an embodiment of this application. The audio system 300 in Figure 3 includes a wearable audio device 310 and other terminal devices 320.

[0151] Referring to Figure 3, the other terminal device 320 includes a first communication module 321 and an instruction recognition module 322; the wearable audio device 310 includes an instruction execution module 313, an audio acquisition module 312, and a second communication module 311.

[0152] For example, the audio acquisition module 312 can output the acquired audio signal 10 to the second communication module 311; subsequently, the second communication module 311 outputs the audio signal 10 to the first communication module 321. Then, the first communication module 321 outputs the audio signal 10 to the instruction recognition module 322; next, the instruction recognition module 322 can recognize the control instruction 11 and output the control instruction 11 to the first communication module 321; subsequently, the first communication module 321 outputs the control instruction 11 to the second communication module 311. The second communication module 311 outputs the control instruction 11 to the instruction execution module 313; then, the instruction execution module 313 executes the control operation corresponding to the control instruction 11.

[0153] For example, the wearable audio device 310 in Figure 3 can be smart glasses, and other terminal devices 320 can be mobile phones, tablets, etc.

[0154] Figure 4 is a structural block diagram of another audio system 400 according to an embodiment of this application. The audio system 400 in Figure 4 includes a wearable audio device 410 and other terminal devices 420.

[0155] Referring to Figure 4, the other terminal device 420 includes a first communication module 421, an instruction recognition module 422, and a first instruction execution module 423; the wearable audio device 410 includes a second instruction execution module 413, an audio acquisition module 412, and a second communication module 411.

[0156] For example, the audio acquisition module 412 can output the acquired audio signal 10 to the second communication module 411; subsequently, the second communication module 411 outputs the audio signal 10 to the first communication module 421. Then, the first communication module 421 outputs the audio signal 10 to the instruction recognition module 422; next, the instruction recognition module 422 can identify the first control instruction 111 and the second control instruction 112; then, the instruction recognition module 422 outputs the second control instruction 112 to the second communication module 421 and outputs the first control instruction 111 to the first instruction execution module 423; the first instruction execution module 423 executes the control operation corresponding to the first control instruction 111. Subsequently, the second communication module 421 outputs the second control instruction 112 to the second instruction execution module 413; then, the second instruction execution module 413 executes the control operation corresponding to the second control instruction 112.

[0157] For example, the wearable audio device 410 in Figure 4 can be a head-mounted display, and other terminal devices 420 can be mobile phones, tablets, etc.

[0158] In other words, the audio systems in Figures 2 to 4 all include wearable audio devices and other terminal devices. The wearable audio devices collect audio signals, the other terminal devices identify control commands, and the wearable audio devices and / or other terminal devices execute the control operations corresponding to the control commands.

[0159] Figure 5 is a structural block diagram of another audio system 500 according to an embodiment of this application. The audio system 500 in Figure 5 includes a wearable audio device 510 and other terminal devices 520.

[0160] Referring to Figure 5, the other terminal device 520 includes a first communication module 521 and an instruction execution module 522; the wearable audio device 510 includes an instruction recognition module 513, an audio acquisition module 512, and a second communication module 511.

[0161] For example, the audio acquisition module 512 can output the acquired audio signal 10 to the instruction recognition module 513; then, the instruction recognition module 513 can recognize the control instruction 11 and output the control instruction 11 to the second communication module 511; subsequently, the second communication module 511 outputs the control instruction 11 to the first communication module 511. Then, the first communication module 511 outputs the control instruction 11 to the instruction execution module 522; next, the instruction execution module 522 executes the control operation corresponding to the control instruction 11.

[0162] For example, the wearable audio device 510 in Figure 5 can be a head-mounted display, and other terminal devices 520 can be mobile phones, tablets, etc.

[0163] Figure 6 is a structural block diagram of another audio system 600 according to an embodiment of this application. The audio system 600 in Figure 6 includes a wearable audio device 610 and other terminal devices 620.

[0164] Referring to Figure 6, the other terminal device 620 includes a first communication module 621 and a first instruction execution module 622; the wearable audio device 610 includes a second instruction execution module 614, an instruction recognition module 613, an audio acquisition module 612, and a second communication module 611.

[0165] For example, the audio acquisition module 612 can output the acquired audio signal 10 to the instruction recognition module 613; then, the instruction recognition module 613 can identify the first control instruction 111 and the second control instruction 112, and output the first control instruction 111 to the second communication module 611 and the second control instruction 112 to the second instruction execution module 614; subsequently, the second instruction execution module 614 can execute the control operation corresponding to the second control instruction 112; and the second communication module 611 outputs the first control instruction 111 to the first communication module 611. Then, the first communication module 611 outputs the first control instruction 111 to the first instruction execution module 622; next, the instruction execution module 622 executes the control operation corresponding to the first control instruction 111.

[0166] For example, the wearable audio device 610 in Figure 6 can be a head-mounted display, and other terminal devices 620 can be mobile phones, tablets, etc.

[0167] In other words, the audio systems in Figures 5 and 6 all include wearable audio devices and other terminal devices. The wearable audio devices collect audio signals and identify control commands; the wearable audio devices and other terminal devices execute the control operations corresponding to the control commands, or the other terminal devices execute the control operations corresponding to the control commands.

[0168] Figure 7 is a structural block diagram of another audio system 700 according to an embodiment of this application. The audio system 700 in Figure 7 only includes a wearable audio device 710.

[0169] Referring to Figure 7, the wearable audio device 710 includes an instruction execution module 711, an instruction recognition module 713, and an audio acquisition module 712.

[0170] For example, the audio acquisition module 712 can output the acquired audio signal 10 to the instruction recognition module 713; then, the instruction recognition module 713 can recognize the control instruction 11 and output the control instruction 11 to the instruction execution module 711; then, the instruction execution module 711 executes the control operation corresponding to the control instruction 11.

[0171] For example, the wearable audio device 710 in Figure 7 can be a head-mounted display, a smartwatch, etc.

[0172] It should be noted that the first communication module and the second communication module in Figures 2 to 6 can be either wireless communication modules or wired communication modules. When both the first and second communication modules are wireless communication modules, the wearable audio device can establish wireless communication connections with other terminal devices, such as Bluetooth communication, WIFI communication, etc.

[0173] It should be noted that the audio acquisition module in Figures 2 to 7 can be a bone conduction pickup unit (such as a vibration pickup unit, VPU) or a microphone pickup unit, and this application does not limit it.

[0174] For example, one of the concepts of this application is to use a target sound to control the audio system; that is, the control command recognized by the command recognition module is essentially a control command corresponding to the audio signal of the target sound. The control operations performed by the command execution module (or the first command execution module or the second command execution module) may include, but are not limited to: starting / stopping / specific operations of system functions (such as Bluetooth, wireless, positioning, voice assistant, etc.), starting / stopping / specific operations of applications (such as audio applications, broadcast applications, audiobook applications, navigation applications, settings applications, etc.), audio volume control (such as increasing or decreasing volume), music switching, music playback / pause, making / answering / hanging up calls, switching sound effect modes, and turning on or off certain channels of a multi-channel playback device (such as the bone conduction channel in a bone conduction headset), etc.

[0175] For example, the target sound can refer to the sound produced by mechanical contact of a user's body part. Mechanical contact can include, but is not limited to, tapping, collision, and friction. User body parts can include, but are not limited to, bones, teeth, and skin. Mechanical contact of a user's body part can include mechanical contact between other objects and the user's body part, or it can include mechanical contact between user body parts themselves.

[0176] For example, the target sound may include at least one of the following: a sound produced by striking bones, a sound produced by teeth colliding, or a sound produced by rubbing skin.

[0177] For example, the sound produced by teeth colliding can include the sound of a user biting their teeth.

[0178] For example, the sounds produced by tapping bones can include: the sounds produced by a user tapping facial bones, the sounds produced by a user tapping around the ear, the sounds produced by a user tapping the skull, or the sounds produced by a user tapping the wrist / back of the hand. Facial bones can include, for example, the cheekbone, frontal bone, and jawbone.

[0179] For example, the sound produced by rubbing the skin can include the sound produced by a user rubbing their ear.

[0180] For example, the audio system can pre-set or user-defined preset audio sequences and corresponding control instructions. A set of control instructions may include one sub-control instruction, or two or more logically related sub-control instructions. When a set of control instructions includes two or more logically related sub-control instructions, the preset audio sequence corresponding to that set of control instructions may include two or more sub-sequences, and each sub-control instruction in the set of control instructions may correspond to one sub-sequence.

[0181] For example, the same preset audio sequence can correspond to a set of control commands in different scenarios (or, a set of control commands in different scenarios can correspond to the same preset audio sequence). For instance, preset audio sequence 1 corresponds to a set of control commands for the audio playback scenario: pause audio; a set of control commands for the audio pause playback scenario: play audio; and a set of control commands for the incoming call scenario: answer the phone. Thus, when the phone receives a call, the target sound corresponding to preset audio sequence 1 can be emitted to answer the call; when music is playing on the phone, the target sound corresponding to preset audio sequence 1 can be emitted to pause music playback; and when music is paused on the phone, the target sound corresponding to preset audio sequence 1 can be emitted to play music.

[0182] For example, the preset audio sequences corresponding to a set of control commands can also be different in different scenarios. For instance, answering a phone call corresponds to preset audio sequence 1, and hanging up a phone call corresponds to preset audio sequence 2.

[0183] For example, each set of control commands may correspond to one or more preset audio sequences.

[0184] In one possible implementation, the preset audio sequence can be composed of audio signals of long and short tones of the same target sound.

[0185] For example, consider the sound produced by a user biting their teeth (hereinafter referred to as dentition). A long dentition can be a dentition where the interval between the next dentition and the next dentition is greater than a first threshold (e.g., 0.5 seconds); a short dentition can be a dentition where the interval between the next dentition and the next dentition is less than a second threshold (e.g., 0.2 seconds). (It should be noted that the last dentition in the audio signal can be identified as a short dentition.) The preset audio sequences and corresponding control commands pre-set by the audio system or user-defined settings can be shown in Table 1 below:

[0186] Table 1

[0187] In Table 1, “·” represents the audio signal of a short sibilant sound.

[0188] “━” represents the audio signal of a long sibilant sound.

[0189] The symbol “→” indicates that the two subsequences are related, and that they correspond to two logically related control commands. For example, in “··━→·”, the first subsequence “··━” corresponds to decreasing the volume, and the second subsequence “·” corresponds to stopping the volume decrease. Similarly, in “━··→·”, the first subsequence “━··” corresponds to increasing the volume, and the second subsequence “·” corresponds to stopping the volume increase.

[0190] Referring to Table 1, the preset audio sequence "..." corresponds to a set of control commands that may include only one control command. This single control command can be pausing audio in an audio playback scenario, playing audio in an audio paused playback scenario, or answering a call in an incoming call scenario. This can be understood as the three sets of control commands—pausing audio, playing audio, and answering a call—corresponding to the same preset audio sequence. The preset audio sequence "..." also corresponds to a set of control commands that may include only one control command. This single control command can be switching to the next track in an audio playback scenario or starting call recording in a call scenario. This can be understood as the two sets of control commands—switching to the next track and starting call recording—corresponding to the same preset audio sequence. The preset audio sequence "..." can include two control commands: decreasing volume and stopping decreasing volume. The preset audio sequence "..." can include two control commands: increasing volume and stopping increasing volume. The preset audio sequence "..." can include only one control command: activating call privacy mode.

[0191] In one possible implementation, the preset audio sequence can be composed of audio signals of at least two target sounds.

[0192] For example, the audio sequence may be a combination of audio signals of any two or three target sounds, such as the sound of tapping bones, the sound of teeth colliding, or the sound of rubbing skin.

[0193] For example, preset audio sequences and corresponding control commands can be pre-set by the audio system or user-defined settings, as shown in Table 2 below:

[0194] Table 2

[0195] In Table 2, “□” represents the audio signal of the sound produced by teeth colliding; “△” represents the audio signal of the sound produced by striking bones; and “☆” represents the audio signal of the sound produced by rubbing the skin.

[0196] For example, the audio sequence may be a combination of audio signals of any two or more of the following target sounds: a long tone of sound produced by tapping bones, a short tone of sound produced by tapping bones, a long tone of sound produced by teeth colliding, a short tone of sound produced by teeth colliding, a long tone of sound produced by rubbing skin, or a short tone of sound produced by rubbing skin.

[0197] For example, preset audio sequences and corresponding control commands can be pre-set by the audio system or user-defined settings, as shown in Table 3 below:

[0198] Table 3

[0199] In Table 2, “□” represents the audio signal of a long tone produced by teeth colliding; “■” represents the audio signal of a short tone produced by teeth colliding; “△” represents the audio signal of a long tone produced by striking bones; “▲” represents the audio signal of a short tone produced by striking bones; “☆” represents the audio signal of a long tone produced by rubbing skin; and “★” represents the audio signal of a short tone produced by rubbing skin.

[0200] The following example illustrates how users can customize preset audio sequences and corresponding control commands through other terminal devices, using a preset audio sequence composed of a combination of long and short sibilants.

[0201] Figures 8A to 8E are schematic diagrams of the mobile phone interface according to embodiments of this application.

[0202] Referring to Figure 8A(1), exemplarily, 801 is the main interface of the mobile phone, which includes one or more controls, including but not limited to: application icons (e.g., application icon of browser application, application icon of settings application 802), etc.

[0203] Referring again to Figure 8A(1), when the user clicks the application icon 802 of the settings application, the mobile phone can respond to the user's operation and display the settings interface 803, as shown in Figure 8A(2). For example, the settings interface 803 may include one or more controls, including but not limited to: settings option 1, settings option 2, ..., settings selection 7 and command and audio sequence settings option 804, etc., which are not limited in this application.

[0204] Referring to Figure 8A(2), when a user needs to set a control command or a preset audio sequence corresponding to the control command, they can click the command and audio sequence setting option 804. The mobile phone can respond to the user's operation and display the command and audio sequence setting interface 805, as shown in Figure 8B. For example, the command and audio sequence setting interface 805 may include one or more controls, including but not limited to: audio sequence recording option 806, control list management option 807, etc. This application does not limit this.

[0205] Referring to Figure 8B, when a user needs to customize the preset audio sequence corresponding to the recording control command, they can click the audio sequence recording option 806. The mobile phone can respond to the user's operation and display the personalized audio sequence recording interface 808, as shown in Figure 8C(1). The personalized audio sequence recording interface 808 may include one or more controls, such as the start recording option. The user can click the start recording option, and the mobile phone can respond to the user's operation and display the recording interface 809; and activate the microphone in the mobile phone or the microphone of a wearable audio device connected to the mobile phone. For example, the recording interface 809 may include one or more controls, such as the end recording option. Afterwards, the user can generate sound by vibrating their teeth to realize the preset audio sequence corresponding to the custom setting control command; for example, the user can emit three long sibilants, one short sibilant, two long sibilants and one short sibilant in sequence. After the user finishes recording, they can click the end recording option in the recording interface 809, and the mobile phone can respond to the user's operation and display the audio sequence detection result interface 810, as shown in Figure 8D(1).

[0206] Referring to Figure 8D(1), the audio sequence detection result interface 810 may include one or more controls, including but not limited to: audio sequence detection result, re-recording option, multiple control command options, etc. The user can select a control command option, and the control command corresponding to the control command option will be used as the control command bound to the audio sequence entered by the user; for example, if the user clicks to open the call privacy mode, the mobile phone will respond to the user's operation and display the binding result interface 811, as shown in Figure 8D(2). For example, the binding result interface 811 may include one or more controls, including but not limited to: audio sequence detection result, re-recording option, target control command option (i.e., open call privacy mode option), confirmation option, recording end option, and continue recording option, etc.

[0207] Referring to Figure 8D(2), when the user confirms that the recorded audio sequence meets their requirements and that the control command bound to the recorded audio sequence meets their requirements, they can click the confirmation option. The mobile phone can respond to the user's operation and bind the recorded audio sequence to the target control command. When the user confirms that they no longer need to record an audio sequence, they can click the end recording option. The mobile phone can respond to the user's operation and display the control list interface 812, as shown in Figure 8E.

[0208] Referring to Figure 8E(1), the control list interface 812 may include one or more controls, such as management list options, multiple audio sequences, and multiple control commands. Users can view control commands and their associated preset audio sequences in the control list interface 812. In addition, users can click on the management list option, and the phone can respond to the user's operation by displaying the management interface 813, as shown in Figure 8E(2). The management interface 813 includes one or more controls, including but not limited to: multiple re-recording options, multiple change command options, and multiple delete options. When a user needs to re-record an audio sequence, they can click on the re-recording option below the audio sequence, and the phone can respond to the user's operation by displaying the personalized audio sequence recording interface 808, as shown in Figure 8C(1). When a user needs to change the control command corresponding to an audio sequence, they can click on the change command option below the control command corresponding to the audio sequence, and the phone can respond to the user's operation by displaying multiple control command options or editing options; users can select and click on any control command option as needed, or enter control commands in the editing options. When a user needs to delete an audio sequence and its corresponding control command, they can click the delete option below the control command for that audio sequence. The phone will then respond to the user's action and delete the audio sequence and its corresponding control command.

[0209] Referring again to Figure 8D, when the user is not satisfied with the recorded audio sequence, they can click the re-record option on the audio sequence detection result interface 810 or the binding result interface 811. The mobile phone can respond to the user's operation and display the personalized audio sequence recording interface 808, as shown in Figure 8C(1).

[0210] Referring again to Figure 8B, the user can click the control list management option 807. The mobile phone can respond to the user's operation and display the management interface 813, as shown in Figure 8E(2). After that, the user can view and manage the audio sequence and corresponding control commands entered by the user in the management interface 813.

[0211] It should be noted that users can also enter audio sequences and manage / view control lists through other entry points (such as headphone applications, Bluetooth device settings interfaces, etc.), and this application does not impose any restrictions on this.

[0212] For example, when the wearable audio device changes, the control commands also change accordingly to adapt to different user needs. For instance, when the wearable audio device is headphones, the audio system can add control commands such as skipping tracks, pausing, previous track, next track, increasing volume, and decreasing volume, along with corresponding preset audio sequences, to the control list. When the wearable audio device is a head-mounted display, the audio system can remove the headphone-related control commands from the control list and add control commands corresponding to the head-mounted display, such as confirm, return, and return to the main menu.

[0213] In other words, an audio sequence can be composed of audio signals generated by repeated mechanical contact of a user's body parts. This combination method makes the audio sequence contain richer information, that is, the audio sequence has diversity. The diversity of audio sequences means that the corresponding control commands are also diverse, which allows users to perform more diverse control over the audio system.

[0214] Furthermore, this application allows users to set preset audio sequences according to their own usage habits, making it easier for users to quickly master how to control the audio system through target sounds. This application also allows users to set corresponding preset audio sequences for frequently used control commands according to their own usage habits, facilitating subsequent quick control of the audio system.

[0215] Based on Figures 2 to 7, the process of audio system recognizing control commands will be explained below.

[0216] Figure 9 is a schematic diagram of a control process 900 according to an embodiment of this application.

[0217] S901, acquires audio signals.

[0218] For example, the audio acquisition module in a wearable audio device can be kept on all the time, so that the audio acquisition module can acquire audio signals in real time or periodically.

[0219] For example, when the audio acquisition module is a microphone unit, the audio signal acquired by the audio acquisition module may include an audio signal obtained through the bone conduction path of sound generated by mechanical contact with a user's body parts, and an audio signal obtained through the air conduction path of sound generated by mechanical contact with a user's body parts. When the audio acquisition module is a VPU unit, the audio signal acquired by the audio acquisition module may be an audio signal obtained through the bone conduction path of sound generated by mechanical contact with a user's body parts.

[0220] For example, when it is determined that there is an audio signal of the target sound in a certain audio signal of the collected audio signal, the audio signal can be output to the instruction recognition module, and the instruction recognition module executes S902.

[0221] S902 identifies the control command corresponding to the audio signal of the target sound in the audio signal. The target sound is the sound produced by mechanical contact of a part of the user's body.

[0222] For example, after receiving the audio signal, the instruction recognition module can identify the control instruction corresponding to the target sound audio signal in the audio signal and output the identified control instruction to the instruction execution module (or, the first instruction execution module and the second instruction execution module), which then executes S903. When the instruction recognition module does not identify the control instruction, the instruction recognition module can execute S902 for the next audio signal obtained from the audio acquisition module (which contains the target sound audio signal).

[0223] In one possible implementation, the instruction recognition module can identify the audio sequence corresponding to the audio signal of the target sound in the audio signal segment; then, the instruction recognition module can compare the audio sequence with a preset audio sequence (audio system settings and / or user-defined audio sequences) to determine the control instruction corresponding to the audio signal of the target sound.

[0224] In one possible approach, the instruction recognition module can first extract the audio features corresponding to the target sound's audio signal in the audio signal segment; then compare these audio features with the audio features of a preset audio sequence to determine the control instruction corresponding to the target sound's audio signal.

[0225] It should be understood that when the audio system is the type shown in Figures 2, 3, 5, and 7, the control commands generated by the command recognition module are of one type. These control commands are executed by the command execution module of the wearable audio device, or by the command execution module of another terminal device. When the audio system is the type shown in Figure 4 or 6, the control commands generated by the command recognition module include two types: a first control command and a second control command. The second control command is executed by the second command execution module of the wearable audio device, and the first control command is executed by the first control command execution module of another terminal device.

[0226] The first control instruction can be understood as a control instruction for other terminal devices, and the second control instruction can be understood as a control instruction for wearable audio devices.

[0227] S903 executes the control operation corresponding to the control command.

[0228] For example, the instruction execution module (or the first instruction execution module and the second instruction execution module) can execute the control operation corresponding to the control instruction.

[0229] For example, suppose the audio system 200 includes a wearable audio device 210 (headphones) and other terminal devices 220 (mobile phones), with music playing in the headphones; if the control command is to switch to the next song, the mobile phone's command execution module can switch to the next song after receiving the control command. In this scenario, the target sound could be the sound produced by the user biting their teeth.

[0230] For example, suppose the audio system 300 includes a wearable audio device 310 that is smart glasses, and other terminal devices 320 that are mobile phones; if the control command is to display temperature, then after receiving the control command, the command execution module of the smart glasses can display the temperature data collected by the temperature sensor of the smart glasses on the lens. In this scenario, the target sound could be the sound produced by the user tapping around the ear or the sound produced by the user tapping the facial bones.

[0231] For example, suppose the audio system 400 includes a wearable audio device 410, which is a VR headset, and other terminal devices 420, which are mobile phones; the VR headset is playing a movie. If the first control command is to pause and the second control command is to open the display menu, then after receiving the second control command, the second command execution module of the VR headset can display the menu on the VR headset's screen; after receiving the first control command, the first command execution module of the mobile phone can pause the movie. In this scenario, the target sound could be a sound produced by the user tapping around the ear, a sound produced by the user tapping facial bones, or a sound produced by the user tapping the skull.

[0232] For example, suppose the audio system 600 includes a wearable audio device 610 that is a smartwatch. If the control command is to display heart rate, the smartwatch's command execution module can display the heart rate after receiving the control command. In this scenario, the target sound could be the sound produced by the user tapping their wrist / back of their hand.

[0233] In other words, this application controls the audio system by picking up the sounds generated by the mechanical contact of user body parts. Since most audio systems have built-in sound pickup capabilities, no special hardware is required. Therefore, the software algorithm for recognizing control commands based on the sounds generated by the mechanical contact of user body parts is easily portable and can be ported to most audio systems. Secondly, the sounds generated by the mechanical contact of user body parts are diverse, including sounds from striking bones, teeth colliding, and skin rubbing. This allows users to customize the way the sounds generated by the mechanical contact of user body parts are used to trigger control commands, offering a high degree of customization and meeting individual user needs. Thirdly, in addition to using hands, users can also operate the system using other methods such as biting their teeth. Therefore, this application is applicable to various scenarios, such as sports or when both hands are occupied, making its application scenarios rich.

[0234] Furthermore, there are many combinations of sounds generated by mechanical contact with user body parts. Therefore, compared with the prior art, this application has more ways to control the audio system and more flexible control operation of the audio system.

[0235] Furthermore, compared to traditional methods of controlling audio systems, this application controls the audio system through sound generated by mechanical contact of the user's body parts, which is less likely to attract attention, offers greater privacy, and is suitable for certain special scenarios such as conferences.

[0236] In addition, compared to other methods of generating target sounds, generating target sounds by biting your teeth does not require the use of your hands, making it more private and convenient.

[0237] Taking the audio system 200 in Figure 2 as an example, the process of the instruction recognition module recognizing control instructions will be explained in detail.

[0238] Figure 10A is a schematic diagram of the system framework of another audio system 1000 according to an embodiment of this application. Figure 10A is a detailed representation of the audio system 200 in Figure 2.

[0239] The audio system 1000 in Figure 10A may include a wearable audio device 1010 and other terminal devices 1020. The wearable audio device 1010 may include a VPU unit 1012 and a second short-range transmission module 1011; the other terminal devices 1020 may include a first short-range transmission module 1021, a preprocessing module 1024, an instruction recognition module 1022, and an instruction execution module 1023.

[0240] For example, VPU unit 1012 is an example of audio acquisition module 212, second short-range transmission module 1011 is an example of second communication module 211, and first short-range transmission module 1021 is an example of first communication module 221.

[0241] For example, the preprocessing module 1024 may include a noise suppression module 10241 and a sound event detection module 10242. The instruction recognition module 1022 may include a sound intent recognition module 10221, an audio sequence detection module 10222, and an instruction determination module 10223.

[0242] Figure 10B is a schematic diagram of another control process 1000 according to an embodiment of this application.

[0243] S1001, acquires audio signals.

[0244] For example, the VPU unit 1012 of the wearable audio device 1010 can acquire the audio signal 10 and output the audio signal 10 to the second short-range transmission module 1011. Then, the second short-range transmission module 1011 can output the audio signal 10 to the first short-range transmission module 1021. Subsequently, the second short-range transmission module 1021 can output the audio signal 10 to the preprocessing module 1024.

[0245] Initially, the VPU unit 1012 samples according to a preset sampling period. In this case, at the beginning of the preset sampling period, the VPU unit 1012 acquires an audio signal of a first preset duration (which can be referred to as the first audio signal for ease of description), sends the first audio signal to the preprocessing module 1024, and enters a sleep state until the next preset sampling period arrives. After receiving a first switching instruction from the preprocessing module 1024, the VPU unit 1012 switches from sampling according to the preset sampling period to real-time sampling. In this case, after acquiring one frame of audio signal (wherein, each frame of audio signal acquired in real-time sampling is referred to as the second audio signal), the VPU unit 1012 sends the second audio signal to the preprocessing module 1024. After receiving a second switching instruction from the preprocessing module 1024, the VPU unit 1012 switches from real-time sampling to sampling according to the preset sampling period. In this case, after acquiring a segment of the first audio signal, the VPU unit 1012 sends the first audio signal to the preprocessing module 1024 and enters a sleep state until the next preset sampling period arrives.

[0246] The preset sampling period and the first preset duration can both be set according to requirements.

[0247] S1002 performs noise suppression on the audio signal.

[0248] For example, the noise suppression module 10241 in the preprocessing module 1024 performs noise suppression on a segment of audio signal 10 (a first audio signal or a second audio signal) (wherein, when the audio signal 10 is a second audio signal, noise suppression can be performed on a segment of the second audio signal).

[0249] For example, the noise suppression performed by the noise suppression module 10241 may include, but is not limited to, noise reduction, filtering, and anti-interference. Noise suppression can be implemented using traditional algorithms or neural networks; this application does not impose any limitations on this. The noise suppression module 10241 performs noise suppression on the audio signal 10 to obtain a noise-suppressed audio signal 12. Then, the noise suppression module 10241 can output the noise-suppressed audio signal 12 to the sound event detection module 10242.

[0250] S1003 performs sound event detection on the noise-suppressed audio signal.

[0251] For example, the sound event detection module 10242 can perform sound event detection on the noise-suppressed audio signal 12 to detect whether there is a target sound event in the noise-suppressed audio signal 12, that is, to determine whether the noise-suppressed audio signal 12 contains an audio signal of the target sound.

[0252] For example, the sound event detection module 10242 can be implemented using traditional algorithms or using neural networks (such as Convolutional Recurrent Neural Networks (CRNNs)). This application does not impose any restrictions on this. For example, this application can optimize the training (also known as fine-tuning) of a pre-trained neural network (which may include multiple neural networks, each used to identify a certain target sound event; or, the neural network may also be a single neural network used to identify different target sound events of multiple users) as follows: collect training data (training data associated with the current user can be entered when the neural network is used for the first time; or, historical data associated with the current user can be collected at some time after use); wherein, the training data includes two types: the first type of training data includes multiple sets of training data, each set of training data including a training audio signal containing the target sound (generated by the current user) and a label (such as "1"); the second type of training data includes multiple sets of training data, each set of training data including a training audio signal not containing the target sound (generated by the current user) and a label (such as "0"). Next, the neural network can be trained using each set of training data. For example, the training audio signal from one set of training data can be input into the neural network, which will process the audio signal and output a sound event detection result. If the sound event detection result is different from the label in that set of training data, the weights of the neural network can be adjusted so that the output sound event detection result matches the label in that set of training data. If the sound event detection result matches the label in that set of training data, there is no need to adjust the weights of the neural network, and the next set of training data can be used to train the neural network.

[0253] For example, the sound event detection module 10242 can determine whether the audio signal of the target sound is detected from the noise-suppressed audio signal 12 based on the sound event detection result. For instance, if the sound event detection result is "1", it can be determined that the audio signal of the target sound is detected from the noise-suppressed audio signal 12; if the sound event detection result is "0", it can be determined that the audio signal of the target sound is not detected from the noise-suppressed audio signal 12.

[0254] For example, when the noise-suppressed audio signal 12 is a noise-suppressed first audio signal, when the audio signal of the target sound is detected for the first time, the sound event detection module 10242 can store the audio segment in the noise-suppressed first audio signal from the moment the target sound begins to appear to the end of the noise-suppressed first audio signal, and send a first switching instruction to the VPU unit 1012 to instruct the VPU unit 1012 to perform real-time sampling.

[0255] For example, if the noise-suppressed audio signal 12 is a noise-suppressed second audio signal, the second audio signal can be stored, and storage of the noise-suppressed second audio signal can be stopped if no target sound audio signal is detected within a second preset time period. Next, the sound event detection module 10242 can send a second switching instruction to the VPU unit 1012 to instruct the VPU unit 1012 to sample according to a preset sampling period; and the sound event detection module 10242 can output the audio segments from the moment the target sound begins to appear in the stored noise-suppressed first audio signal to the end of the noise-suppressed first audio signal, as well as one or more segments of noise-suppressed second audio signals (these are all noise-suppressed audio signals, corresponding to 13 in Figure 10A), to the sound intent recognition module 10221.

[0256] S1004, When the audio signal of the target sound is detected from the noise-suppressed audio signal, the sound intent is recognized from the noise-suppressed audio signal.

[0257] For example, the target sound in the noise-suppressed audio signal 13 may be a sound generated by the user in order to control the audio system, or it may be a sound generated by the user's misoperation, such as the sound of the user biting their teeth when eating, the sound of the user biting their teeth when speaking, or the sound of the user accidentally biting their teeth. Therefore, the voice intent recognition module 10221 needs to perform voice intent recognition on the audio signal of the target sound in the noise-suppressed audio signal 13 to determine whether the user has the intention to control the audio system.

[0258] For example, the voice intent recognition module 10221 can be implemented using a traditional algorithm or a neural network (such as an attention-based neural network), and this application does not impose any limitations on this. For example, this application can pre-optimize and train a pre-trained neural network, and the process can be as follows: collect training data; wherein, the training data includes two types: the first type of training data includes multiple sets of training data, each set of training data including a training audio signal containing the audio signal of the target sound generated by the user in order to control the audio system and a label (such as "1"); the second type of training data includes multiple sets of training data, each set of training data including a training audio signal containing the audio signal of the target sound generated by the user's erroneous operation and a label (such as "0"). Then, each set of training data can be used to train the neural network. For example, the training audio signal in a set of training data is input into the neural network, and the neural network processes the training audio signal and outputs a voice intent recognition result; if the voice intent recognition result is different from the label in the set of training data, the weights of the neural network can be adjusted so that the voice intent recognition result output by the neural network is the same as the label in the training data. If the voice intent recognition result is the same as the label in the training data set, there is no need to adjust the weights of the neural network; the next set of training data can be used to train the neural network.

[0259] For example, the voice intent recognition module 10221 can determine whether the audio signal of the target sound in the noise-suppressed audio signal 13 has a control intent based on the voice intent recognition result. For instance, if the voice intent recognition result is "1", it can be determined that the audio signal of the target sound in the noise-suppressed audio signal 13 has a control intent; if the voice intent recognition result is "0", it can be determined that the audio signal of the target sound in the noise-suppressed audio signal 13 does not have a control intent.

[0260] When the voice intent recognition module 10221 determines that the target sound audio signal in the noise-suppressed audio signal 13 has a control intent, the voice intent recognition module 10221 can output the voice intent recognition result 14 of "having a control intent" to the audio sequence detection module 10222. When the voice intent recognition module 10221 determines that the target sound audio signal in the noise-suppressed audio signal 13 does not have a control intent, the voice intent recognition module 10221 can end the current process and continue to perform voice intent recognition on the next segment of noise-suppressed audio signal 13.

[0261] S1005, when the control intention is identified from the noise-suppressed audio signal, an audio sequence is detected from the noise-suppressed audio signal.

[0262] For example, when the audio sequence detection module 10222 receives the sound intent recognition result 14 of "having control intent", it determines that the control intent has been identified from the noise-suppressed audio signal 13, and the audio sequence detection module 10222 can detect the audio sequence from the noise-suppressed audio signal 13.

[0263] For example, the audio sequence detection module 10222 can be implemented using a traditional algorithm or a neural network, and this application does not limit it in this regard.

[0264] When the audio sequence detection module 10222 is implemented using a traditional algorithm, taking the sound produced by a user biting their teeth as an example, the process of the audio sequence detection module 10222 detecting the audio sequence can be as follows: detect the interval time between each dentate sound and the next dentate sound; when the interval time between the dentate sound and the next dentate sound is greater than a first threshold, determine that the dentate sound is a long dentate sound and record it with a first identifier (such as "1"); when the interval time between the dentate sound and the next dentate sound is less than a second threshold, determine that the dentate sound is a short dentate sound and record it with a second identifier (such as "0").

[0265] In one possible implementation, when the interval between the sibilant sound and the next sibilant sound is greater than a second threshold but less than a first threshold, the sibilant sound is determined to be an ambiguous sound and recorded using a third identifier (such as "2"). Thus, the audio sequence output by the audio sequence detection module 10222 consists of a first identifier, a second identifier, and a third identifier (which may not be present). For example, assuming the audio signal includes the audio signal of the sound generated by the user to "turn on call privacy mode," the audio sequence output by the audio sequence detection module 10222 may be [1,1,1,0,1,1,0], [1,1,2,0,1,1,0], or [1,1,1,2,1,1,0], etc.

[0266] In one possible approach, when the interval between the sibilant sound and the next sibilant sound is greater than a second threshold and less than a first threshold, if the time interval is closer to the second threshold, then a second identifier (e.g., "0") is used for recording; if the time interval is closer to the first threshold, then a first identifier (e.g., "1") is used for recording in the audio sequence. Thus, the audio sequence output by the audio sequence detection module 10222 consists of the first identifier and the second identifier. For example, assuming the audio signal includes the audio signal of the sound generated by the user to "turn on call privacy mode," then the audio sequence output by the audio sequence detection module 10222 is {[1,1,1,0,1,1,0].

[0267] In one possible approach, when the interval between the first and second sibilants is greater than a second threshold but less than a first threshold, if the interval is closer to the second threshold, a fourth identifier (e.g., "3") is used for recording; if the interval is closer to the first threshold, a fifth identifier (e.g., "4") is used for recording. Thus, the audio sequence output by the audio sequence detection module 10222 consists of a first identifier, a second identifier, a fourth identifier (which may not be present), and a fifth identifier (which may not be present). For example, assuming the audio signal includes the audio signal generated by the user to "enable call privacy mode," the audio sequence output by the audio sequence detection module 10222 might be [1,1,1,0,1,1,0], [1,1,3,0,1,1,0], or [1,1,1,4,1,1,0], etc.

[0268] It should be understood that this application does not impose any restrictions on the audio sequence output by the audio sequence detection module 10222.

[0269] When the audio sequence detection module 10222 is implemented using a neural network, the noise-suppressed audio signal 13 can be input into the neural network, processed by the neural network, and output as an audio sequence.

[0270] For example, this application can pre-train a pre-trained neural network for optimization training. The process can be as follows: Collect training data; wherein the training data includes multiple sets, each set of training data may include a training audio signal containing an audio signal of the target sound generated by the user for controlling the audio system, and annotations (such as audio sequences). Then, each set of training data can be used to train the neural network. For example, the training audio signal from one set of training data is input into the neural network, which processes the training audio signal and outputs an audio sequence. If the audio sequence output by the neural network is different from the audio sequence in that set of training data, the weights of the neural network can be adjusted so that the audio sequence output by the neural network is the same as the audio sequence in that set of training data. If the audio sequence output by the neural network is the same as the audio sequence in that set of training data, there is no need to adjust the weights of the neural network, and the next set of training data can be used to train the neural network.

[0271] Subsequently, when the audio sequence detection module 10222 detects an audio sequence, it can output the audio sequence 15 to the instruction determination module 10223, which then executes S1006. When the audio sequence detection module 10222 does not detect an audio sequence, the current process can end; and the detection of the audio sequence can continue from the next noise-suppressed audio signal 13 (with control intent).

[0272] S1006, When an audio sequence is detected from a noise-suppressed audio signal, the control command corresponding to the audio sequence is determined.

[0273] For example, when the instruction determination module 10223 receives the audio sequence 15, it determines that the audio sequence has been detected from the noise-suppressed audio signal, and the instruction determination module 10223 can determine the control instruction corresponding to the audio sequence.

[0274] For example, the instruction determination module 10223 can be implemented using a traditional algorithm or a neural network, and this application does not limit it in this regard.

[0275] When the instruction determination module 10223 is implemented using a conventional algorithm, if the audio sequence consists only of a first identifier and a second identifier, the instruction determination module 10223 can determine the target audio sequence from multiple preset audio sequences (where long sibilants in the preset audio sequences can be represented by the first identifier, and short sibilants can be represented by the second identifier) ​​based on the audio sequence; for example, a preset audio sequence identical to the audio sequence can be determined as the target audio sequence. If the target audio sequence is found from multiple preset audio sequences, the control instruction corresponding to the target audio sequence is determined as the control instruction 11 corresponding to the audio signal of the target sound; and the instruction determination module 10223 outputs the control instruction 11 to the instruction execution module 1023. If the target audio sequence is not found from multiple preset audio sequences, the instruction determination module 10223 can end the current process; and continue to determine the control instruction corresponding to the next noise-suppressed audio signal 13.

[0276] When the audio sequence includes a third identifier, the third identifier can be replaced with the first identifier to obtain a first audio sequence. Then, based on the first audio sequence, a target audio sequence is determined from multiple preset audio sequences. If the target audio sequence is found from the multiple preset audio sequences, the control instruction corresponding to the target audio sequence is determined as control instruction 11 (referred to as control instruction 1) corresponding to the audio signal of the target sound. Afterward, the third identifier can be replaced with the second identifier to obtain a second audio sequence. Then, based on the second audio sequence, the target audio sequence is determined from multiple preset audio sequences. If the target audio sequence is found from the multiple preset audio sequences, the control instruction corresponding to the target audio sequence is determined as control instruction 11 (referred to as control instruction 2) corresponding to the audio signal of the target sound. If the interval between the sibilant corresponding to the third identifier and the next sibilant is closer to the first threshold, the instruction determination module 10223 outputs the control instruction 1 (11) to the instruction execution module 1023; if the interval between the sibilant corresponding to the third identifier and the next sibilant is closer to the second threshold, the instruction determination module 10223 outputs the control instruction 2 (11) to the instruction execution module 1023. If the instruction determination module 10223 can only determine the target audio sequence from multiple preset audio sequences based on the first audio sequence, the instruction determination module 10223 outputs the control instruction 1 (11) to the instruction execution module 1023. If the instruction determination module 10223 can only determine the target audio sequence from multiple preset audio sequences based on the second audio sequence, the instruction determination module 10223 outputs the control instruction 2 (11) to the instruction execution module 1023. If the instruction determination module 10223 fails to determine the target audio sequence from multiple preset audio sequences based on either the first audio sequence or the second audio sequence, the instruction determination module 10223 can end the current process and continue to determine the control instruction corresponding to the next noise-suppressed audio signal 13.

[0277] When the instruction determination module 10223 is implemented using a neural network, the audio sequence detected from the noise-suppressed audio signal 13 can be directly input into the neural network for processing, and the neural network can output control instructions or other information. For example, if the neural network outputs a control instruction, the instruction determination module 10223 can send the control instruction 11 output by the neural network to the instruction execution module 1023; if the neural network outputs other information, the instruction determination module 10223 can end the current process and continue to determine the control instruction corresponding to the next segment of the noise-suppressed audio signal 13.

[0278] S1007 executes the control operation corresponding to the control command.

[0279] For example, the instruction execution module 1023 can execute the control operation corresponding to the control instruction.

[0280] Figure 11 is a schematic diagram of the system framework of another audio system 1100 according to an embodiment of this application. Figure 11 is a detailed representation of the audio system 200 in Figure 2.

[0281] The audio system 1100 in Figure 11 may include a wearable audio device 1110 and other terminal devices 1120. The wearable audio device 1110 may include a microphone unit 1112 and a second short-range transmission module 1111; the other terminal devices 1120 may include a first short-range transmission module 1121, a preprocessing module 1124, an instruction recognition module 1122, and an instruction execution module 1123.

[0282] For example, microphone unit 1112 is an example of audio acquisition module 212, second short-range transmission module 1111 is an example of second communication module 211, and first short-range transmission module 1121 is an example of first communication module 221.

[0283] For example, the instruction recognition module 1122 can be implemented by an end-to-end neural network. In this way, the preprocessing module 1124 inputs the noise-suppressed audio signal 13 into the end-to-end neural network, which processes the signal and outputs control instructions or other information. For example, if the end-to-end neural network outputs a control instruction, the instruction recognition module 1122 can send the control instruction 11 output by the end-to-end neural network to the instruction execution module 1023; if the neural network outputs other information, the instruction recognition module 1122 can end the current process and send an instruction to the preprocessing module 1124, which instructs the preprocessing module 1124 to preprocess the next segment of audio signal; this process is repeated.

[0284] For example, when the audio system determines that a user has issued a control command through a target sound, for some control commands, the audio system needs to receive confirmation from the user before executing the corresponding control operation; for other control commands, the audio system can directly execute the corresponding control operation and notify the user of the result after execution. This makes the interaction between the audio system and the user more user-friendly.

[0285] In one possible implementation, the audio system also outputs confirmation information indicating whether the control operation should be performed; upon receiving confirmation from the user, the control operation is executed. This reduces the risk of accidental operation.

[0286] For example, the audio system can display a confirmation screen indicating whether to perform the control operation. This confirmation screen includes a denial option and a confirmation option. In response to the user's click on the confirmation option, the control operation is performed. As shown in Figure 12A, when the control command is "Turn on airplane mode," the phone can display a confirmation screen 1201 indicating whether to turn on airplane mode before performing this operation. Confirmation screen 1201 can include a denial option 1203 and a confirmation option 1202. When the user confirms turning on airplane mode, they can click the confirmation option 1202, and the phone can respond to the user's action by turning on airplane mode. When the user denies turning on airplane mode, they can click the denial option 1201, and the phone can respond to the user's action by not turning on airplane mode. As another example, when the control command is "Hang up the phone," the phone can display a confirmation screen indicating whether to hang up the phone before performing this operation. This confirmation screen can include a denial option and a confirmation option. When the user confirms hanging up the phone, they can click the confirmation option, and the phone can respond to the user's action by hanging up the phone. When a user confirms that they do not want to hang up, they can click the "deny" option, and the phone will respond to the user's action and not hang up.

[0287] For example, the audio system can display a confirmation screen indicating whether to execute the control operation. This confirmation screen includes a countdown and a denial option. If no user click on the denial option is received within a third preset time period (e.g., 10 seconds), the control operation is executed. If a user click on the denial option is received within the third preset time period (e.g., 10 seconds), the control operation is refused in response to the user's click on the confirmation option. In this way, even if the user's hands are occupied, it is still possible to confirm whether to execute the control operation.

[0288] For example, an audio system can play a confirmation voice indicating whether a control operation should be performed; in response to the user's confirmation voice, the control operation is performed.

[0289] For example, vibration can be used to provide confirmation of whether the control operation has been performed. For instance, a wearable audio device (such as headphones) could vibrate to provide confirmation. Alternatively, other terminal devices (such as mobile phones) could vibrate to provide confirmation.

[0290] For example, confirmation of whether the control operation was performed could be provided in the form of a ringtone / sound effect. Alternatively, confirmation could be provided by a wearable audio device (such as headphones) playing a ringtone / sound effect.

[0291] It should be understood that the above-mentioned feedback methods can be combined.

[0292] In one possible implementation, the audio system also outputs the result of the control operation. This allows the user to be informed whether the control of the audio system was successful.

[0293] The execution result of the control operation can include success or failure.

[0294] For example, an audio system can play the voice of the result of a control operation.

[0295] For example, the audio system can display the execution result of the control operation. As shown in Figure 12B, when the control command is "switch to the next song", the mobile phone can display the song switching result interface 1204 after switching to the next song, and display "Song switching successful" on the song switching result interface 1204.

[0296] For example, the execution result can be fed back in the form of vibration. For instance, a wearable audio device (such as headphones) can vibrate to provide feedback on the execution result. Alternatively, other terminal devices (such as mobile phones) can also vibrate to provide feedback. The vibration pattern for successful execution can differ from that for failed execution; for example, a successful execution might be characterized by a single long vibration, while a failed execution might be characterized by multiple short vibrations.

[0297] For example, the execution result can be fed back in the form of a ringtone / sound effect. Alternatively, the execution result can be fed back by a ringtone / sound effect played by a wearable audio device (such as headphones).

[0298] It should be understood that the vibration pattern for feedback execution results is different from the vibration pattern for confirmation information regarding whether the control operation was performed; and the sound effect / ringtone for feedback execution results may be different from the sound effect / ringtone for confirmation information regarding whether the control operation was performed.

[0299] It should be understood that the above-mentioned feedback methods can be combined.

[0300] In one possible implementation, the audio system also outputs a cancellation message indicating whether the control operation has been undone; upon receiving confirmation of cancellation from the user, the control operation is undone. This way, the user can also cancel the control operation performed on the audio system via a target sound.

[0301] For example, the audio system can display an undo interface for the control operation, which includes an undo option. In response to the user clicking the undo option, the control operation is undone. As shown in Figure 12C, when the control command is "increase volume," the phone can display an undo interface 1205 during the "increase volume" operation. This undo interface 1205 can include an undo option 1206. When the user needs to undo the "increase volume" operation, they can click the undo option 1206. The phone can then respond to the user's action and undo the "increase volume" operation, restoring the volume to its state before the "increase volume" operation was performed.

[0302] For example, the audio system can play a voice prompt asking whether to cancel the control operation; in response to the user's confirmation to cancel the control operation, the execution of the control operation is canceled.

[0303] For example, vibration could be used to provide feedback on whether the control operation has been cancelled. For instance, a wearable audio device (such as headphones) could vibrate to provide feedback on whether the control operation has been cancelled. Alternatively, other terminal devices (such as mobile phones) could vibrate to provide feedback on whether the control operation has been cancelled.

[0304] For example, feedback could be provided in the form of a ringtone / sound effect to indicate whether the control operation has been undone. Alternatively, feedback could be provided by a wearable audio device (such as headphones) playing a ringtone / sound effect.

[0305] It should be understood that the vibration pattern for feedback on whether to cancel the execution of the control operation is different from the vibration pattern for feedback on whether to confirm the execution of the control operation and the execution result; and the sound effect / ringtone for feedback on whether to cancel the execution of the control operation is different from the sound effect / ringtone for feedback on whether to confirm the execution of the control operation and the execution result.

[0306] It should be understood that the above-mentioned feedback methods can be combined.

[0307] In one possible implementation, the audio system further includes outputting a stop message indicating whether the control operation should be stopped; upon receiving confirmation from the user, the control operation is stopped. This allows the user to control the audio system at any time and stop their control operations via a target sound.

[0308] For example, a stop operation interface can be displayed, which includes a stop option. In response to the user clicking the stop option, the control operation stops. As shown in Figure 12D, when the control command is "increase volume," the phone can display a stop operation interface 1207 during the "increase volume" operation. This stop operation interface 1207 can include a stop option 1208. When the user needs to stop the "increase volume" operation, they can click the stop option 1208. The phone can then respond to the user's action and stop the "increase volume" operation, thus maintaining the volume at the state it was in when the "increase volume" operation was stopped.

[0309] For example, a voice prompt can be played asking whether to stop the control operation; in response to the user's confirmation to stop the control operation, the control operation is stopped.

[0310] For example, vibration can be used to provide feedback indicating whether the control operation has been stopped. For instance, a wearable audio device (such as headphones) could vibrate to provide this feedback. Alternatively, other terminal devices (such as mobile phones) could vibrate to provide this feedback.

[0311] For example, feedback on whether to stop the control operation can be provided in the form of a ringtone / sound effect. For instance, a wearable audio device (such as headphones) can play a ringtone / sound effect to provide feedback on whether to stop the control operation.

[0312] It should be understood that the vibration pattern of the stop message indicating whether the control operation has been stopped is different from the vibration pattern of the cancellation message indicating whether the control operation has been revoked, the vibration pattern of the confirmation message indicating whether the control operation has been executed, and the vibration pattern of the execution result. The sound effect / ringtone of the stop message indicating whether the control operation has been stopped may also be different from the sound effect / ringtone of the cancellation message indicating whether the control operation has been revoked, the sound effect / ringtone of the confirmation message indicating whether the control operation has been executed, and the sound effect / ringtone of the execution result.

[0313] It should be understood that the above-mentioned various forms of feedback can be combined.

[0314] In one possible implementation, the audio system also provides feedback on the execution of the control operation through vibration. For example, if the control command is "increase volume," the phone will generate continuous vibration through the headphones when it begins volume control, alerting the user that the volume control has taken effect. Afterwards, feedback can be provided in any of the above forms to indicate whether the volume increase operation has stopped. When the user issues a control command to stop increasing the volume, the phone stops increasing the volume and stops vibrating.

[0315] For example, this application can also simplify all operations of any terminal device into four control operations: confirm, return, previous, and next. Then, four target sound audio signals can be set, each corresponding to a control command for one of these four control operations. This allows for application in special usage scenarios such as when hands are occupied by the back of the device or when a person with disabilities uses the terminal.

[0316] Figure 13 is a schematic diagram of another mobile phone interface provided in an embodiment of this application.

[0317] Figure 13 shows a camera interface 1301, which includes one or more controls, including but not limited to: aperture setting options, night scene mode options, portrait mode options, photo mode options, video mode options, professional mode options, more options, preview options, photo options, camera orientation options, focus adjustment options, AI photo options, flash options, settings options, etc.

[0318] Assuming the current selection box 1302 is in the aperture shooting option: If a confirmation command is received, the phone responds by displaying the aperture settings interface. If a "previous" control command is received, the phone can move selection box 1302 to the focus adjustment option. If a "next" control command is received, the phone can move selection box 1302 to the night mode option. If a "back" control command is received, the phone can exit the camera interface 1301 or move selection box 1302 back to the shooting option.

[0319] Optionally, when the selection box is moved to a certain option, the option can be read aloud; this makes it easier for people with poor eyesight to use the terminal device. For example, in Figure 13, after the phone moves the selection box 1302 to the focus adjustment option, it can read aloud the voice of "focus adjustment"; after the phone moves the selection box 1302 to the night scene mode option, it can read aloud the voice of "night scene mode".

[0320] For example, this application also provides a control device that can be used to execute the above-described control process; wherein, the control device may include:

[0321] An audio acquisition module is used to acquire audio signals;

[0322] The instruction recognition module is used to identify the control instruction corresponding to the audio signal of the target sound in the audio signal, wherein the target sound is the sound generated by mechanical contact of a user's body part;

[0323] The instruction execution module is used to execute the control operation corresponding to the control instruction.

[0324] It should be noted that the control device can be a wearable audio device or a device deployed on a wearable audio device. Alternatively, the control device can be deployed within an audio system, or the control device itself can be an audio system, where the audio acquisition module is deployed on the wearable audio device, and the command recognition module and command execution module are deployed on other terminal devices; or, the command recognition module is deployed on other terminal devices, and the command execution module is deployed on both the wearable audio device and other terminal devices. Alternatively, the audio acquisition module and command recognition module are deployed on the wearable audio device, and the command recognition module is deployed on other terminal devices.

[0325] In one example, FIG14 shows a schematic block diagram of an apparatus 1400 according to an embodiment of the present application. The apparatus 1400 may include a processor 1401 and a transceiver 1402, and optionally, a memory 1403.

[0326] The various components of device 1400 are coupled together via bus 1404, which includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 1404 in the figure.

[0327] Optionally, the memory 1403 can be used to store instructions from the foregoing method embodiments. The processor 1401 can be used to execute the instructions in the memory 1403, control the transceiver 1402 to receive signals, and control the transceiver 1402 to transmit signals.

[0328] Device 1400 may be an electronic device or a chip of an electronic device as described in the above method embodiments. The electronic device may be a wearable electronic device or other terminal device.

[0329] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0330] This application also provides a chip including one or more interface circuits and one or more processors; the one or more processors receive or transmit data through the one or more interface circuits, and when the one or more processors execute computer instructions, the steps of the above-described related method steps that implement the method in the above embodiments are executed. The interface circuit is a transceiver 1402.

[0331] This application also provides a computer-readable storage medium storing computer instructions. When these computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the methods described in the above embodiments. Exemplarily, the computer-readable storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0332] This application also provides a computer program product comprising computer instructions that, when executed by a computer or processor, cause the computer to perform the aforementioned steps to implement the methods described in the above embodiments. Exemplarily, the computer program product may be stored in random access memory (RAM), flash memory, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, read-only optical discs (CD-ROMs), or any other form of storage medium known in the art.

[0333] In this embodiment, the electronic device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0334] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0335] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0336] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0337] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0338] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An audio system, characterized in that, The audio system is used for Acquire audio signals; The control command corresponding to the audio signal of the target sound in the audio signal is identified, wherein the target sound is the sound generated by mechanical contact of a part of the user's body; Execute the control operation corresponding to the control command.

2. The audio system according to claim 1, characterized in that, The control command corresponding to the audio signal that identifies the target sound in the audio signal includes: Detect the audio sequence corresponding to the audio signal of the target sound; The control command is identified based on the audio sequence.

3. The audio system according to claim 2, characterized in that, The step of identifying the control command based on the audio sequence includes: Based on the audio sequence, a target audio sequence is determined from multiple preset audio sequences. Each preset audio sequence corresponds to one or more sets of control commands, and the multiple sets of controls correspond to different scenarios. The control command corresponding to the target audio sequence is determined to be the control command corresponding to the audio signal of the target sound.

4. The audio system according to claim 3, characterized in that, The preset audio sequence is either user-defined or set by the audio system.

5. The audio system according to claim 3 or 4, characterized in that, The control commands corresponding to the preset audio sequence are either user-defined or set by the audio system.

6. The audio system according to any one of claims 2 to 5, characterized in that, The audio sequence is a combination of long and short audio signals of the same target sound.

7. The audio system according to any one of claims 2 to 5, characterized in that, The audio sequence is a combination of audio signals of at least two target sounds.

8. The audio system according to any one of claims 1 to 7, characterized in that, The target sound includes at least one of the following: the sound produced by striking bones, the sound produced by teeth colliding, or the sound produced by rubbing skin.

9. The audio system according to any one of claims 1 to 8, characterized in that, The audio system is also used to output the execution result of the control operation.

10. The audio system according to any one of claims 1 to 9, characterized in that, The audio system is also used to output confirmation information on whether to execute the control operation; after receiving confirmation from the user, it executes the control operation.

11. The audio system according to any one of claims 1 to 10, characterized in that, The audio system is also used to output cancellation information indicating whether to cancel the execution of the control operation; upon receiving a confirmation cancellation instruction from the user, the execution of the control operation is cancelled.

12. The audio system according to any one of claims 1 to 10, characterized in that, The audio system is also used to output stop information indicating whether to stop executing the control operation; upon receiving confirmation from the user to stop execution, the control operation is stopped.

13. The audio system according to any one of claims 1 to 12, characterized in that, The audio system includes wearable audio devices and other terminal devices; The wearable audio device is used to collect audio signals and transmit the audio signals; The other terminal devices are used to identify the control command corresponding to the audio signal of the target sound in the audio signal; and execute the control operation corresponding to the control command.

14. The audio system according to any one of claims 1 to 12, characterized in that, The audio system includes wearable audio devices and other terminal devices; The wearable audio device is used to collect audio signals and transmit the audio signals; The other terminal devices are used to identify the control command corresponding to the audio signal of the target sound in the audio signal and send the control command; The wearable audio device is also used to execute control operations corresponding to the control commands.

15. The audio system according to any one of claims 1 to 12, characterized in that, The audio system includes wearable audio devices and other terminal devices; The wearable audio device is used to collect audio signals and transmit the audio signals; The other terminal devices are used to identify the control command corresponding to the audio signal of the target sound in the audio signal, the control command including a first control command and a second control command; execute the control operation corresponding to the first control command and send the second control command; The wearable audio device is also used to execute control operations corresponding to the second control command.

16. The audio system according to any one of claims 1 to 12, characterized in that, The audio system includes wearable audio devices and other terminal devices; The wearable audio device is used to collect audio signals; identify control commands corresponding to the audio signals of target sounds in the audio signals; and send the control commands. The other terminal devices are used to execute the control operations corresponding to the control commands.

17. The audio system according to any one of claims 1 to 12, characterized in that, The audio system includes wearable audio devices and other terminal devices; The wearable audio device is used to collect audio signals; identify control commands corresponding to the audio signals of target sounds in the audio signals, the control commands including a first control command and a second control command; execute the control operation corresponding to the second control command and send the first control command; The other terminal devices are used to execute the control operations corresponding to the first control command.

18. The audio system according to any one of claims 1 to 12, characterized in that, The audio system includes wearable audio devices. The wearable audio device is used to collect audio signals; identify control commands corresponding to the audio signals of target sounds in the audio signals; and execute control operations corresponding to the control commands.

19. A wearable audio device, characterized in that, The wearable audio device is used for Acquire audio signals; The control command corresponding to the audio signal of the target sound in the audio signal is identified, wherein the target sound is the sound generated by mechanical contact of a part of the user's body; The control command is sent to other terminal devices, and the control command is used to instruct the other terminal devices to perform corresponding control operations.

20. A terminal device, characterized in that, The terminal device is used for Receive audio signals sent by wearable audio devices; The control command corresponding to the audio signal of the target sound in the audio signal is identified, wherein the target sound is the sound generated by mechanical contact of a part of the user's body; Execute the control operation corresponding to the control command.

21. A control method, characterized in that, The method includes: Acquire audio signals; The control command corresponding to the audio signal of the target sound in the audio signal is identified, wherein the target sound is the sound generated by mechanical contact of a part of the user's body; Execute the control operation corresponding to the control command.

22. A command recognition method for wearable devices, characterized in that, The method includes: Acquire audio signals; The control command corresponding to the audio signal of the target sound in the audio signal is identified, wherein the target sound is the sound generated by mechanical contact of a part of the user's body; The control command is sent to other terminal devices, and the control command is used to instruct the other terminal devices to perform corresponding control operations.

23. A control method applied to a terminal device, characterized in that, The method includes: Receive audio signals sent by wearable audio devices; The control command corresponding to the audio signal of the target sound in the audio signal is identified, wherein the target sound is the sound generated by mechanical contact of a part of the user's body; Execute the control operation corresponding to the control command.

24. A control device, characterized in that, The device includes: An audio acquisition module is used to acquire audio signals; The instruction recognition module is used to identify the control instruction corresponding to the audio signal of the target sound in the audio signal, wherein the target sound is the sound generated by mechanical contact of a user's body part; The instruction execution module is used to execute the control operation corresponding to the control instruction.

25. A chip, characterized in that, It includes one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the steps of the method as described in any one of claims 21 to 23 are performed.

26. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed on a computer or processor, causes the computer or processor to perform the method as described in any one of claims 21 to 23.

27. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a computer or processor, cause the steps of the method as described in any one of claims 21 to 23 to be performed.

Images