Human-computer interaction control method and device for true wireless earphone and storage medium

Abstract

The present disclosure relates to a human-computer interaction control method and device of a true wireless earphone and a storage medium. The human-computer interaction control method of the true wireless earphone (TWS earphone) comprises: in response to the TWS earphone being connected with a device, the optical sensor detects a hand gesture of a user, and identifies an interaction function corresponding to the detected hand gesture; and according to the interaction function, a control instruction is generated, the control instruction being used to control the TWS earphone and / or the device to perform the interaction function corresponding to the hand gesture. The human-computer interaction control method of the true wireless earphone provided in the present disclosure enables the user to control the TWS earphone and / or the device connected with the TWS earphone to perform the interaction function through the hand gesture, thereby improving the convenience and accuracy of the human-computer interaction implementation.

Description

Technical Field

[0001] This disclosure relates to the field of headphone technology, and in particular to human-computer interaction control methods, devices and storage media for true wireless headphones. Background Technology

[0002] With the development of wireless headphone technology, the application of true wireless stereo (TWS) headphones is becoming increasingly common. TWS headphones can establish a wireless connection with electronic devices such as mobile phones, thereby enabling human-computer interaction control functions such as music play / pause, noise cancellation on / off, and previous / next track, enriching the human-computer interaction experience of TWS headphones and increasing their ease of use.

[0003] In related technologies, pressure-sensitive or touch-based controls are commonly used for human-computer interaction control of TWS earbuds. However, this contact-based control method is prone to problems such as accidental pressure touches and touch control failures. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a human-computer interaction control method, device and storage medium for true wireless earphones.

[0005] According to a first aspect of the present disclosure, a human-computer interaction control method for a TWS earphone is provided, applied to a TWS earphone, wherein the TWS earphone is provided with an optical sensor, the method comprising:

[0006] The system confirms that the TWS earphones are connected to the device. The optical sensor detects the user's gesture and identifies the interactive function corresponding to the detected gesture. Based on the interactive function, a control command is generated, which is used to control the TWS earphones and / or the device to execute the interactive function corresponding to the gesture.

[0007] In one embodiment, generating control instructions based on the interaction function includes: in response to the TWS earphones connecting to a single device, and the TWS earphones including a first TWS earphone and a second TWS earphone, controlling the first TWS earphone that detects a user gesture to generate a first control instruction corresponding to the interaction function, and / or controlling the second TWS earphone that detects a user gesture to generate a second control instruction corresponding to the interaction function; the first control instruction and the second control instruction may be the same as or different from each other.

[0008] In one embodiment, generating control instructions based on the interactive function includes: in response to the TWS earphone being connected to multiple devices, and the TWS earphone including a first TWS earphone and a second TWS earphone, determining a first device controlled by the first TWS earphone and a second device controlled by the second TWS earphone; controlling the first TWS earphone to generate a third control instruction for controlling the first device and / or the TWS earphone to perform the interactive function, and / or controlling the second TWS earphone to generate a fourth control instruction for controlling the second device and / or the TWS earphone to perform the interactive function.

[0009] In one embodiment, determining a first device controlled by the first TWS earphone and a second device controlled by the second TWS earphone includes: determining the time when the TWS earphone establishes a connection with each of the plurality of devices; determining the device that first establishes a connection with the TWS earphone as the first device controlled by the first TWS earphone; and determining the device that establishes a connection with the TWS earphone after the first device as the second device controlled by the second TWS earphone.

[0010] In one embodiment, before identifying the interactive function corresponding to the detected gesture, the human-computer interaction control method of the TWS earphone further includes: determining that the TWS earphone is in a wearing state.

[0011] In one embodiment, identifying the interactive function corresponding to the detected gesture includes: determining the interactive function corresponding to the detected gesture based on a predefined relationship between the gesture and the interactive function; wherein the interactive function includes one or a combination of the following functions: volume increase / decrease control function for TWS earphones and / or devices playing media files; control function for devices playing previous or next media files; control function for devices playing media files forward or backward; and control function for devices taking photos.

[0012] In one embodiment, the human-computer interaction control method of the TWS earphone further includes: in response to determining, based on the interaction function type, that the device connected to the TWS earphone performs the interaction function, sending the control command to the device connected to the TWS earphone.

[0013] According to a second aspect of the present disclosure, a human-computer interaction control device for TWS earphones is provided, applied to TWS earphones, wherein the TWS earphones are provided with an optical sensor, and the device includes:

[0014] The processing unit is used to determine that the TWS earphones are connected to the device, and the optical sensor detects the user's gesture and identifies the interactive function corresponding to the detected gesture;

[0015] The generation unit is configured to generate control instructions based on the interactive function, the control instructions being used to control the TWS earphones and / or the device to perform the interactive function corresponding to the gesture.

[0016] In one implementation, the generation unit generates control commands based on the interactive function in the following manner:

[0017] In response to the TWS earphones being connected to a single device, and the TWS earphones including a first TWS earphone and a second TWS earphone, the first TWS earphone that detects the user's gesture is controlled to generate a first control command corresponding to the interactive function, and / or the second TWS earphone that detects the user's gesture is controlled to generate a second control command corresponding to the interactive function;

[0018] The first control command may be the same as or different from the second control command.

[0019] In one implementation, the generation unit generates control commands based on the interactive function in the following manner:

[0020] In response to the TWS earphones being connected to multiple devices, and the TWS earphones including a first TWS earphone and a second TWS earphone, a first device controlled by the first TWS earphone and a second device controlled by the second TWS earphone are determined;

[0021] The first TWS earphone is controlled to generate a third control command for controlling the first device and / or the TWS earphone to perform the interactive function, and / or the second TWS earphone is controlled to generate a fourth control command for controlling the second device and / or the TWS earphone to perform the interactive function.

[0022] In one embodiment, the generation unit determines the first device controlled by the first TWS earphone and the second device controlled by the second TWS earphone in the following manner:

[0023] The time at which the TWS earphones establish a connection with each of the multiple devices is determined; the device that first establishes a connection with the TWS earphones is determined as the first device controlled by the first TWS earphones; the device that establishes a connection with the TWS earphones after the first device is determined as the second device controlled by the second TWS earphones.

[0024] In one embodiment, the processing unit is further configured to: determine that the TWS earphone is in a wearing state before recognizing the interactive function corresponding to the detected gesture.

[0025] In one embodiment, the processing unit identifies the interaction function corresponding to the detected gesture in the following manner: based on a predefined relationship between the gesture and the interaction function, the interaction function corresponding to the detected gesture is determined;

[0026] The interactive functions include one or a combination of the following:

[0027] Volume control function for TWS earphones and / or devices playing media files;

[0028] The device has control functions for playing the previous or next media file;

[0029] The device has controls for forward and backward playback of media files.

[0030] The device has a photo-taking control function.

[0031] In one embodiment, the human-computer interaction control device of the TWS earphone further includes a transmitting unit.

[0032] The processing unit is further configured to: determine, based on the interaction function type, that the device connected to the TWS earphones will perform the interaction function; the sending unit is configured to send the control command to the device connected to the TWS earphones.

[0033] According to a third aspect of the present disclosure, a human-computer interaction control device for TWS earphones is provided, comprising:

[0034] Processor; memory used to store processor-executable instructions;

[0035] The processor is configured to execute the human-computer interaction control method for TWS earphones as described in the first aspect or any embodiment of the first aspect.

[0036] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor of a TWS earphone, enable the TWS earphone to perform the human-computer interaction control method of the TWS earphone described in the first aspect or any embodiment of the first aspect.

[0037] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: when TWS earphones are connected to a device and a user's gesture is detected, the interactive function corresponding to the detected gesture is identified, and control instructions for the interactive function corresponding to the gesture are generated. This enables the TWS earphones and / or the device to execute the interactive function corresponding to the gesture. Compared with the contact operation control method, this improves the convenience and accuracy of human-computer interaction.

[0038] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0040] Figure 1 This is a flowchart illustrating a human-computer interaction control method for TWS earphones according to an exemplary embodiment.

[0041] Figure 2 This is a flowchart illustrating another human-computer interaction control method for TWS earphones according to an exemplary embodiment.

[0042] Figure 3 This is a flowchart illustrating a method for generating control commands for TWS earphones according to an exemplary embodiment.

[0043] Figure 4 This is a flowchart illustrating another method for generating control commands for controlling TWS earphones according to an exemplary embodiment.

[0044] Figure 5 This is a flowchart illustrating a method for determining the device controlling each earphone in a TWS earphone according to an exemplary embodiment.

[0045] Figure 6 This is a flowchart illustrating another method for determining the device controlling each earbud in a TWS earphone according to an exemplary embodiment.

[0046] Figure 7 This is a block diagram illustrating a human-computer interaction control device for TWS earphones according to an exemplary embodiment.

[0047] Figure 8 This is a block diagram illustrating a human-computer interaction control device for TWS earphones according to an exemplary embodiment. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0049] The human-computer interaction control method for TWS earphones disclosed herein can be applied to scenarios where users interact with TWS earphones and / or devices connected to the TWS earphones. For example, it can be used in scenarios where users control TWS earphones and / or connected mobile phones, tablets, laptops, TVs, etc., to perform interactive functions.

[0050] In related technologies, TWS earbuds typically feature touch buttons and / or pressure-sensitive components, allowing users to perform pressure-sensitive or touch-based operations to enable various interactive functions. For example, users can use touch controls to control the TWS earbuds and / or the connected device to perform functions such as music play / pause, noise cancellation on / off, and skip to the previous / next track. However, this touch-based control method is prone to issues such as accidental pressure touches, touch malfunctions, and control failures when two devices are connected. For instance, when wearing TWS earbuds, users cannot see the pressure-sensitive detection area, leading to accidental touches. If a user wants to replay the previous track by touching the pressure-sensitive area but accidentally touches the pressure-sensitive area for the next track, the TWS earbuds will control the connected device to play the next track, negatively impacting the user experience.

[0051] In view of this, this disclosure provides a human-computer interaction control method for TWS earphones, in which an optical sensor is installed in the TWS earphone. This optical sensor supports gesture recognition and generation. When the TWS earphone is connected to a device and the optical sensor of the TWS earphone detects a user's gesture, the interaction energy corresponding to the gesture is recognized, and a control command corresponding to the interaction function is generated. This allows the user to control the interactive functions of the TWS earphone and / or the device connected to the TWS earphone through gesture operation without touching or using physical buttons, reducing the possibility of accidental triggering of the TWS earphone's interactive functions.

[0052] Figure 1 This is a flowchart illustrating a human-computer interaction control method for TWS earphones according to an exemplary embodiment, such as... Figure 1 As shown, the human-computer interaction control method for TWS earphones is used in TWS earphones equipped with optical sensors and includes the following steps.

[0053] In step S11, it is determined that the TWS earphones are connected to the device, the optical sensor detects the user's gesture, and identifies the interactive function corresponding to the detected gesture.

[0054] In this embodiment, TWS earphones can connect to devices such as mobile phones, tablets, laptops, and televisions, and provide corresponding audio service functions. Users can interact with the audio service functions through the TWS earphones and the connected devices. The TWS earphones can be understood as the slave device providing the audio service function, and the device connected to the TWS earphones can be understood as the master device providing the audio service function.

[0055] Among them, TWS earbuds and devices can be connected via Bluetooth communication protocol.

[0056] Furthermore, the TWS earbuds are equipped with an optical sensor that can detect and recognize gestures. The process of gesture detection and recognition using the optical sensor can be implemented using relevant technologies, such as infrared light or lasers.

[0057] Furthermore, user gestures can control interactive functions. For example, there is a correspondence between gestures and interactive functions. Once a gesture is recognized, the corresponding interactive function can be determined based on this correspondence.

[0058] In step S12, control instructions are generated based on the interactive function. These control instructions are used to control the TWS earphones and / or devices to perform the interactive function corresponding to the gesture.

[0059] In this embodiment of the disclosure, an optical sensor is placed on the TWS earphone, which can detect the user's gesture operation through the optical sensor. Then, based on the detected gesture, the TWS earphone and / or the device connected to the TWS earphone can be controlled to perform corresponding interactive functions. This eliminates the need to touch the TWS earphone, making the operation more convenient for the user and reducing the possibility of accidental triggering of interactive functions.

[0060] In the human-computer interaction control method for TWS earphones provided in this disclosure, a predefined relationship between gestures and interactive functions can be set, that is, the interactive function corresponding to each gesture can be determined. For example, it can be defined that the interaction function of controlling the volume up / down corresponds to the user's hand moving closer to / away from the TWS earphone, or it can be that the interaction function of controlling the previous / next track of music playback corresponds to the user's hand moving closer to / away from the TWS earphone. Furthermore, it can also be that different movement directions of gestures within a certain range can be defined to realize different interactive functions of the TWS earphone controlling the device to take photos, record videos, etc.

[0061] Among them, the interactive functions of TWS earphones, such as taking photos and recording videos, can be set with different gestures for the same function. For example, waving your hand forward within a certain range takes a photo, waving your hand backward takes a quick shot, and keeping your hand at the earphone position records video. Removing your hand from the earphone position stops recording.

[0062] In one example, in this embodiment of the disclosure, the interaction function corresponding to a gesture can be determined by a predefined relationship between the gesture and the interaction function. The interaction function may be, for example, one or a combination of the following: volume control function for TWS earphones and / or devices playing media files; control function for playing the previous or next media file; control function for playing media files forward or backward; and control function for taking a photo.

[0063] In another example, once the interaction function corresponding to the user's gesture is determined, the device connected to the TWS earphones can be chosen to perform the interaction function based on its type, and a control command can be sent to the device connected to the TWS earphones. For example, if the required interaction function is a photo-taking function, since the TWS earphones cannot perform this function, the device connected to the TWS earphones is chosen to perform the photo-taking function, and a photo-taking command can be sent to that device.

[0064] Figure 2 This is a flowchart illustrating another human-computer interaction control method for TWS earphones according to an exemplary embodiment, such as... Figure 2 As shown, the steps include the following.

[0065] In step S21, in response to the TWS earphones connecting to the device and the optical sensor detecting the user's gesture, the interaction function corresponding to the detected gesture is determined based on the predefined relationship between the gesture and the interaction function.

[0066] Among them, the implementation process of step S22 is similar to Figure 1 The execution method of step S12 shown is similar and will not be repeated here.

[0067] In this embodiment of the disclosure, the interaction function corresponding to the detected gesture is determined based on the predefined relationship between gesture and interaction function. This makes it easier for users to use gesture operations to control the interaction function of TWS earphones and / or the devices connected to the TWS earphones, reducing the possibility of accidental triggering of TWS earphone interaction function.

[0068] The TWS earphones involved in the human-computer interaction control method for TWS earphones provided in this embodiment can be a single earphone or two earphones including a left earphone and a right earphone.

[0069] In the case of TWS earbuds including a single earbud, gesture detection, recognition, and generation can be performed through the optical sensor built into the single earbud, and interactive function control commands can be generated through the single earbud.

[0070] The following description, based on the embodiments disclosed herein, focuses on the process of gesture recognition and interactive function control in the case of TWS earphones, which include a left earphone and a right earphone, i.e., two earphones.

[0071] For ease of description, the left earbud of the TWS earphone in this disclosure is referred to as the first TWS earphone, and the right earbud as the second TWS earphone. Of course, it is also possible to refer to the right earbud as the first TWS earphone and the left earbud as the second TWS earphone; this disclosure does not specifically limit this.

[0072] In this embodiment of the disclosure, the first TWS earphone and the second TWS earphone are respectively provided with optical sensors so that the first TWS earphone and the second TWS earphone can independently complete the detection of user gestures.

[0073] In this embodiment, both the first and second TWS earbuds are equipped with built-in optical sensors that support the recognition and generation of various gestures. Furthermore, since TWS earbuds are wearable devices that balance power consumption and portability, the optical sensors built into the first and second TWS earbuds are configured independently, and possess characteristics such as low power consumption, accurate recognition, small size, and low cost.

[0074] In one example of this disclosure, the TWS earbuds, including a first TWS earbud and a second TWS earbud, can be connected to a single device. User gestures are detected and recognized through the first and second TWS earbuds, and human-computer interaction is performed based on these gestures with the connected device.

[0075] Figure 3 This is a flowchart illustrating a method for generating control commands for TWS earphones according to an exemplary embodiment, such as... Figure 3 As shown, it includes the following steps.

[0076] In step S31, it is determined that the TWS earphones are connected to a single device, and the TWS earphones include a first TWS earphone and a second TWS earphone.

[0077] In step S32, the first TWS earphone that detects the user's gesture generates a first control command corresponding to the interactive function, and / or the second TWS earphone that detects the user's gesture generates a second control command corresponding to the interactive function.

[0078] The first control command may be the same as or different from the second control command.

[0079] In one example of this disclosure, it is assumed that the single device connected to the first TWS earphone and the second TWS earphone is a mobile phone. The user uses the first TWS earphone and the second TWS earphone to play and listen to the audio of multimedia videos, such as music, played on the mobile phone. Among them, the predefined interaction function corresponding to "waving forward / waving backward" is "previous track / next track", and the interaction function corresponding to "waving with palm down / waving with palm up" is "volume down / volume up".

[0080] In one embodiment, if one of the TWS earbuds detects a user's gesture, it directly recognizes and sends the control command corresponding to that gesture. For example, if the first TWS earbud detects a "wave forward" gesture, it can generate a first control command to control the phone to play the "next track," thereby controlling the device to switch to the next music track. If the second TWS earbud detects a "wave forward" gesture, it can generate a corresponding control command to control the phone to play the "next track," thereby controlling the device to switch to the next music track. In this case, the gesture detected by the first or second TWS earbud is the same, and the generated first and second control commands are also the same.

[0081] In another embodiment, if the first TWS earphone and the second TWS earphone detect the user's gesture, the first TWS earphone and the second TWS earphone can send control commands respectively based on the detected gesture.

[0082] For example, if the first TWS earbuds detect a "wave forward" gesture from the user, and the second TWS earbuds detect a "wave palm upward" gesture, then the first TWS earbuds can generate a first control command to play the "next track" on the phone, and the second TWS earbuds can generate a second control command to increase the volume on both the TWS earbuds and / or the phone. This allows the device to switch to the next track and increase the volume. In this case, the gestures detected by the first and second TWS earbuds are inconsistent, resulting in different first and second control commands.

[0083] In the human-computer interaction control method for TWS earphones provided in this disclosure, when the TWS earphones are connected to a host device (such as a mobile phone, tablet, laptop, or TV) via Bluetooth, both the first and second TWS earphones have built-in independent low-power, accurate, small-size, and low-cost optical sensor modules that can support various gesture recognition generation. Furthermore, both the first and second TWS earphones can generate control commands independently, such as adjusting volume by moving closer to / away from the earphones within a certain range, controlling the previous / next track, or using gesture scanning within a certain range to control the earphones for taking photos.

[0084] In another scenario of the human-computer interaction control method for TWS earphones provided in this embodiment, the first TWS earphone and the second TWS earphone can be connected to multiple devices, and the user's gestures can be detected through the first TWS earphone and the second TWS earphone, and human-computer interaction can be performed with the connected devices based on the gestures.

[0085] For ease of description, in the embodiments disclosed herein, among the multiple devices connected to the TWS earphones, the device controlled by the first TWS earphone is referred to as the first device, and the device controlled by the second TWS earphone is referred to as the second device.

[0086] In this embodiment of the disclosure, when the first TWS earphone and the second TWS earphone are connected to multiple devices including the first device and the second device, the first device controlled by the first TWS earphone and the second device controlled by the second TWS earphone can be determined among the multiple devices, and the corresponding interactive functions can be performed by controlling the corresponding devices through the first TWS earphone and / or the second TWS earphone.

[0087] Figure 4 This is a flowchart illustrating another method for generating control commands for controlling TWS earphones according to an exemplary embodiment, such as... Figure 4 As shown, it includes the following steps.

[0088] In step S41, in response to the TWS earphones being connected to multiple devices, and the TWS earphones including a first TWS earphone and a second TWS earphone, a first device controlled by the first TWS earphone and a second device controlled by the second TWS earphone are determined.

[0089] In one example of this disclosure, a first device controlled by a first TWS earphone and a second device controlled by a second TWS earphone can be determined based on the time it takes for each of the multiple devices to establish a connection with the TWS earphone.

[0090] Figure 5 This is a flowchart illustrating a method for determining the control device for each earphone in a TWS earphone according to an exemplary embodiment, such as... Figure 5 As shown, it includes the following steps.

[0091] In step S411, the time it takes for the TWS earphones to establish a connection with each of the multiple devices is determined.

[0092] In this embodiment of the disclosure, the time it takes for the TWS earbuds to establish a connection with each device typically varies, so the time it takes for the TWS earbuds to establish a connection with each of the multiple devices can be determined separately. Here, establishing a connection between the TWS earbuds and a device can be understood as the first TWS earbud and / or the second TWS earbud establishing a connection with the device.

[0093] In step S412, the device that first establishes a connection with the TWS earphone is identified as the first device controlled by the first TWS earphone.

[0094] In step S413, the device that establishes a connection with the TWS earphone after the first device is identified as the second device controlled by the second TWS earphone.

[0095] In this embodiment of the disclosure, after identifying a first device controlled by a first TWS earphone and a second device controlled by a second TWS earphone among multiple devices, control commands for controlling the first device and / or the TWS earphone to perform interactive functions can be generated through the first TWS earphone, and control commands for controlling the second device and / or the TWS earphone to perform interactive functions can be generated through the second TWS earphone. That is, the first and second TWS earphones independently generate control commands for their respective control devices to achieve precise control of multiple devices in a multi-device connection scenario, avoiding problems such as control failure under multi-device connection.

[0096] In step S42, the first TWS earphone is controlled to generate a third control command for controlling the first device and / or the TWS earphone to perform interactive functions, and / or the second TWS earphone is controlled to generate a fourth control command for controlling the second device and / or the TWS earphone to perform interactive functions.

[0097] In the human-computer interaction control method for TWS earphones provided in this disclosure embodiment, in a scenario where the first TWS earphone and the second TWS earphone are connected to multiple devices including the first device and the second device, both the first TWS earphone and the second TWS earphone can generate control commands independently and control the first device, the second device, and / or the TWS earphone to perform corresponding interactive functions.

[0098] In this embodiment of the disclosure, the functions that the first TWS earphone and the second TWS earphone control the first device and the second device to perform can be the same, or they can be different. That is, the third control command and the fourth control command can be the same or different.

[0099] In this embodiment of the disclosure, for the same gesture, the first TWS earphone and the second TWS earphone can control the first device and the second device to perform the same function, or they can be different.

[0100] In one example, if the first device and the second device are devices that perform the same type of application functions, the same gesture can perform the same interactive function on both the first device and the second device.

[0101] For example, consider a first device and a second device with photo and video recording capabilities. The interaction function corresponding to the "wave forward" gesture is the photo-taking function. If both the first and second TWS earbuds detect the "wave forward" gesture simultaneously, and the interaction function bound to both earbuds for this gesture is the photo-taking function, then a photo-taking command can be generated through the first TWS earbud to control the first device to take a photo, and a photo-taking command can be generated through the second TWS earbud to control the second device to take a photo.

[0102] In another example, if the first device and the second device are devices that perform different types of application functions, the same gesture can perform different interactive functions on the first device and the second device.

[0103] For example, for a first device such as a mobile phone with photo and video recording capabilities, the "wave forward" gesture corresponds to the photo-taking function. For a second device such as a television with multimedia playback capabilities, the "wave forward" gesture corresponds to the interaction function of switching between playing media videos. In one example, if the first device is performing the photo-taking function and the second device is performing the multimedia playback function, and both the first and second TWS earbuds detect the "wave forward" gesture simultaneously, the first TWS earbuds can control the first device to take a photo, and the second TWS earbuds can control the second device to switch between playing multimedia videos.

[0104] In the human-computer interaction control method for TWS earphones provided in this embodiment, in a scenario where the first TWS earphone and the second TWS earphone are connected to multiple devices including the first device and the second device, the TWS earphone that generates the control command can be determined as the first TWS earphone or the second TWS earphone, and the corresponding controlled device can be determined as the first device or the second device, based on the interaction function corresponding to the gesture.

[0105] In one example, if the interaction function corresponding to the gesture detected by the current TWS earbud is executed by a device controlled by another TWS earbud, then either the current TWS earbud can generate a control command, or the other TWS earbud can generate a corresponding control command, to control the corresponding device to execute the corresponding function. For example, if the function corresponding to the gesture detected by the second TWS earbud can only be executed by the first device, then, on the one hand, the second TWS earbud directly generates a control command and sends it to the first device for execution, controlling the first device to execute the function corresponding to the gesture detected by the second TWS earbud. Alternatively, on the other hand, the second TWS earbud can interact with the first TWS earbud, and the first TWS earbud can generate a control command to control the first device to execute the function corresponding to the gesture detected by the second TWS earbud.

[0106] For example, this applies to a first device such as a mobile phone that has photo and video recording capabilities, and a second device such as a television that does not. When the second TWS earphone determines through the user's gesture that the second device needs to perform a photo-taking function, since the second device does not have a photo-taking function while the first device does, a photo-taking command can be generated through the first TWS earphone to control the first device to perform the photo-taking function corresponding to the user's gesture.

[0107] In the human-computer interaction control method for TWS earphones provided in this disclosure, the system can determine whether the device performing the interaction function is a master device (the device connected to the TWS earphone) or a slave device (the TWS earphone) based on the interaction function type corresponding to the detected gesture. In this disclosure, the device performing the interaction function can be a master device (the device connected to the TWS earphone), a slave device (the TWS earphone), or both, depending on the interaction function type corresponding to the detected gesture.

[0108] For example, if the interaction function is a photo-taking interaction, since the TWS earbuds do not have the ability to execute photo-taking applications, the main device (the device to which the TWS earbuds are connected) is determined to perform this interaction function. As another example, if the interaction function is a volume adjustment interaction, since both the TWS earbuds and the device to which they are connected have the ability to perform volume adjustment, the device to which the TWS earbuds are connected, the TWS earbuds themselves, or both the TWS earbuds and the device to which they are connected can be determined to perform this interaction function.

[0109] In this embodiment of the disclosure, if it is determined that the device connected to the TWS earphone is performing the interaction function based on the interaction function type corresponding to the detected gesture, the TWS earphone also needs to send the generated control command to the device connected to the TWS earphone to control the device connected to the TWS earphone to perform the interaction function corresponding to the gesture.

[0110] Figure 6 This is a flowchart illustrating another method for determining the device controlling each earbud in a TWS earphone, according to an exemplary embodiment. Figure 6 As shown, it includes the following steps.

[0111] In step S51, the device that performs the interactive function is determined according to the interactive function type.

[0112] In step S52, in response to determining that the device connected to the TWS earphone performs the interaction function according to the interaction function type, a control command is sent to the device connected to the TWS earphone.

[0113] The human-computer interaction control method for TWS earphones provided in this disclosure allows the TWS earphones to detect their current state before recognizing the interaction function corresponding to the detected gesture during the human-computer interaction control process. If it is determined that the TWS earphones are currently being worn, the interaction function corresponding to the detected gesture is recognized, thus preventing accidental triggering of TWS earphone interaction functions when the user is not wearing the earphones. Furthermore, it also avoids executing the aforementioned human-computer interaction function process when the TWS earphones are not being worn, preventing wasted power consumption.

[0114] The human-computer interaction control method for TWS earphones provided in this disclosure can control interactive functions of TWS earphones and connected devices in various scenarios. For example, the application scenarios applicable to this disclosure may include:

[0115] Scenario 1: TWS earbuds connect to a main device (such as a mobile phone, tablet, laptop, or TV) via Bluetooth.

[0116] Scenario 2: TWS earbuds connect to two devices via Bluetooth, i.e., dual connection (such as mobile phone, tablet, laptop, TV).

[0117] In this embodiment of the disclosure, both the left and right ears of the TWS earphones are equipped with independent low-power, accurate, small-size and low-cost optical sensor modules, which can support various gesture recognition outputs and accurately perform device interaction functions for different application scenarios.

[0118] For example, in scenario one, both the left and right earbuds of the TWS earbuds can output control commands independently, such as adjusting volume by moving closer to / away from the earbuds within a certain range, or controlling the previous / next track. Gesture scanning within a certain range allows for earbud control of taking photos. Similarly, in scenario two, both the left and right earbuds of the TWS earbuds can output control commands independently, such as adjusting volume by moving closer to / away from the earbuds within a certain range, or controlling the previous / next track. Gesture scanning within a certain range allows for earbud control of taking photos. Furthermore, in this embodiment, the left and right earbuds can be configured to control the order in which devices establish a dual-device connection, such as the left earbud controlling device A (which establishes the connection first) and the right earbud controlling device B (which establishes the connection later), to perform functions such as taking photos, recording videos, and adjusting volume.

[0119] Based on the same concept, this disclosure also provides a human-computer interaction control device for TWS earphones.

[0120] It is understood that the human-computer interaction control device for TWS earphones provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0121] Figure 7 This is a block diagram illustrating a human-computer interaction control device for TWS earphones according to an exemplary embodiment. (Refer to...) Figure 7 The human-computer interaction control device 100 of the TWS earphone includes a processing unit 101 and a generation unit 102.

[0122] Processing unit 101 is used to determine that the TWS earphones are connected to the device, that the optical sensor detects the user's gesture, and that the interaction function corresponding to the detected gesture is identified. Generation unit 102 is used to generate control commands based on the interaction function, and the control commands are used to control the TWS earphones and / or the device to perform the interaction function corresponding to the gesture.

[0123] In one embodiment, the generation unit 102 generates control commands based on the interactive function in the following manner:

[0124] In response to the TWS earphones connecting to a single device, and the TWS earphones including a first TWS earphone and a second TWS earphone, the first TWS earphone that detects the user's gesture generates a first control command corresponding to the interactive function, and / or the second TWS earphone that detects the user's gesture generates a second control command corresponding to the interactive function.

[0125] The first control command may be the same as or different from the second control command.

[0126] In one embodiment, the generation unit 102 generates control commands based on the interactive function in the following manner:

[0127] In response to the TWS earbuds connecting to multiple devices, and the TWS earbuds including a first TWS earbud and a second TWS earbud, a first device controlled by the first TWS earbud and a second device controlled by the second TWS earbud are determined. The first TWS earbud is controlled to generate a third control command for controlling the first device and / or the TWS earbuds to perform interactive functions, and / or the second TWS earbud is controlled to generate a fourth control command for controlling the second device and / or the TWS earbuds to perform interactive functions.

[0128] In one embodiment, the generation unit 102 determines the first device controlled by the first TWS earphone and the second device controlled by the second TWS earphone in the following manner:

[0129] Determine the time when each TWS earbud establishes a connection with one of the multiple devices. The device that first establishes a connection with the TWS earbud is designated as the first device controlled by the first TWS earbud. Devices that establish a connection with the TWS earbud after the first device are designated as the second device controlled by the second TWS earbud.

[0130] In one embodiment, the processing unit 101 is further configured to: determine that the TWS earphone is in a wearing state before recognizing the interactive function corresponding to the detected gesture.

[0131] In one embodiment, the processing unit 101 identifies the interactive function corresponding to the detected gesture in the following manner: based on the predefined relationship between the gesture and the interactive function, the interactive function corresponding to the detected gesture is determined.

[0132] The interactive functions include one or a combination of the following:

[0133] Volume control function for TWS earphones and / or devices playing media files.

[0134] The device has control functions for playing the previous or next media file.

[0135] The device has a function to control the forward or backward playback of media files.

[0136] The device has a photo-taking control function.

[0137] In one embodiment, the human-computer interaction control device 100 of the TWS earphone further includes a transmitting unit 103.

[0138] The processing unit 101 is further configured to: determine, based on the interaction function type, the device connected to the TWS earphones to perform the interaction function. The sending unit 103 is configured to send control commands to the device connected to the TWS earphones.

[0139] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0140] Figure 8 This is a block diagram illustrating a human-computer interaction control device for TWS earphones according to an exemplary embodiment. For example, device 200 may be provided as a Bluetooth earphone. (Refer to...) Figure 8The apparatus 200 includes a processing component 222, which further includes one or more processors, and memory resources represented by memory 232 for storing instructions, such as application programs, that can be executed by the processing component 222. The application programs stored in memory 232 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 222 is configured to execute instructions to perform the methods described above.

[0141] Device 200 may also include a power supply component 226 configured to perform power management of device 200, a wired or wireless network interface 250 configured to connect device 200 to a network, and an input generation (I / O) interface 258. Device 200 may operate on an operating system stored in memory 232, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0142] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 232 including instructions, which can be executed by the processing component 222 of the apparatus 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0143] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0144] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0145] It is further understood that the terms “center,” “longitudinal,” “lateral,” “front,” “rear,” “up,” “down,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” and “outer,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation.

[0146] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0147] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0148] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0149] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A human-computer interaction control method for true wireless TWS earphones, characterized in that, Applied to TWS earphones, the TWS earphones are equipped with optical sensors, and the human-computer interaction control method of the TWS earphones includes: The system confirms that the TWS earphones are connected to the device, and the optical sensor detects the user's gesture and identifies the interactive function corresponding to the detected gesture. Control commands are generated based on the interactive function, and the control commands are used to control the TWS earphones and / or the device to perform the interactive function corresponding to the gesture; The step of generating control commands based on the interactive function includes: In response to the TWS earphones being connected to a single device, and the TWS earphones including a first TWS earphone and a second TWS earphone, the first TWS earphone that detects a user gesture generates a first control command corresponding to the interactive function, and the second TWS earphone that detects a user gesture generates a second control command corresponding to the interactive function, wherein the user gestures detected by the first TWS earphones and the second TWS earphones are the same, and the first control command and the second control command are the same; In response to the TWS earphones being connected to multiple devices, and the TWS earphones including a first TWS earphone and a second TWS earphone, a first device controlled by the first TWS earphone and a second device controlled by the second TWS earphone are determined. Based on the interaction function corresponding to the gesture, the current TWS earphone that generates the control command is determined, and the current TWS earphone generates the control command to control the corresponding device to perform the corresponding function. The step of determining the current TWS earphone that generates the control command based on the interaction function corresponding to the gesture, and having the current TWS earphone generate the control command to control the corresponding device to perform the corresponding function, includes: When the interaction function corresponding to the gesture detected by the second TWS earphone is executed by the first device, the second TWS earphone generates a control command, or the first TWS earphone generates a corresponding control command, to control the first device to execute the corresponding function.

2. The human-computer interaction control method for TWS earphones according to claim 1, characterized in that, The step of determining the current TWS earphone that generates the control command based on the interaction function corresponding to the gesture, and having the current TWS earphone generate the control command to control the corresponding device to perform the corresponding function, includes: The first TWS earphone is controlled to generate a third control command for controlling the first device and / or the TWS earphone to perform the interactive function, and / or the second TWS earphone is controlled to generate a fourth control command for controlling the second device and / or the TWS earphone to perform the interactive function.

3. The human-computer interaction control method for TWS earphones according to claim 1, characterized in that, The first device and the second device determine the connection with the TWS earphone in the following manner: Determine the time it takes for the TWS earphones to establish a connection with each of the multiple devices; The device that first establishes a connection with the TWS earphone is determined as the first device controlled by the first TWS earphone. The device that establishes a connection with the TWS earphone after the first device is identified as the second device controlled by the second TWS earphone.

4. The human-computer interaction control method for TWS earphones according to any one of claims 1 to 3, characterized in that, Before recognizing the interactive function corresponding to the detected gesture, the human-computer interaction control method for TWS earphones further includes: It is determined that the TWS earphones are in a wearing state.

5. The human-computer interaction control method for TWS earphones according to claim 1, characterized in that, Identify the interactive function corresponding to the detected gesture, including: Based on the predefined relationship between gestures and interactive functions, the interactive function corresponding to the detected gesture is determined; The interactive functions include one or a combination of the following: Volume control function for TWS earphones and / or devices playing media files; The device has control functions for playing the previous or next media file; The device has controls for forward and backward playback of media files. The device has a photo-taking control function.

6. The human-computer interaction control method for TWS earphones according to claim 1, characterized in that, The human-computer interaction control method for the TWS earphones also includes: In response to determining, based on the interaction function type, that the device connected to the TWS earphones will perform the interaction function, the control command is sent to the device connected to the TWS earphones.

7. A human-computer interaction control device for true wireless TWS earphones, characterized in that, Applied to TWS earphones, the TWS earphones are equipped with optical sensors, and the human-computer interaction control device of the TWS earphones includes: The processing unit is used to determine that the TWS earphones are connected to the device, and the optical sensor detects the user's gesture and identifies the interactive function corresponding to the detected gesture; The generation unit is configured to generate control instructions based on the interactive function, the control instructions being used to control the TWS earphones and / or the device to perform the interactive function corresponding to the gesture; The generation unit generates control commands based on the interactive function in the following manner: In response to the TWS earphones being connected to a single device, and the TWS earphones including a first TWS earphone and a second TWS earphone, the first TWS earphone that detects a user gesture generates a first control command corresponding to the interactive function, and the second TWS earphone that detects a user gesture generates a second control command corresponding to the interactive function, wherein the user gestures detected by the first TWS earphones and the second TWS earphones are the same, and the first control command and the second control command are the same; In response to the TWS earphones being connected to multiple devices, and the TWS earphones including a first TWS earphone and a second TWS earphone, a first device controlled by the first TWS earphone and a second device controlled by the second TWS earphone are determined. Based on the interaction function corresponding to the gesture, the current TWS earphone that generates the control command is determined, and the current TWS earphone generates the control command to control the corresponding device to perform the corresponding function. Based on the interaction function corresponding to the gesture, the processing unit determines the TWS earphone that generates the control command in the following manner, and the determined TWS earphone generates the control command to control the corresponding device to perform the corresponding function: When the interaction function corresponding to the gesture detected by the second TWS earphone is executed by the first device, the second TWS earphone generates a control command, or the first TWS earphone generates a corresponding control command, to control the first device to execute the corresponding function.

8. The human-computer interaction control device for TWS earphones according to claim 7, characterized in that, Based on the interaction function corresponding to the gesture, the processing unit determines the current TWS earphone that generates the control command in the following manner, and the current TWS earphone generates the control command to control the corresponding device to perform the corresponding function: The first TWS earphone is controlled to generate a third control command for controlling the first device and / or the TWS earphone to perform the interactive function, and / or the second TWS earphone is controlled to generate a fourth control command for controlling the second device and / or the TWS earphone to perform the interactive function.

9. The human-computer interaction control device for TWS earphones according to claim 7, characterized in that, The generation unit determines the connection between the first device and the TWS earphones, and the connection between the second device and the TWS earphones, in the following manner: Determine the time it takes for the TWS earphones to establish a connection with each of the multiple devices; The device that first establishes a connection with the TWS earphone is determined as the first device controlled by the first TWS earphone. The device that establishes a connection with the TWS earphone after the first device is identified as the second device controlled by the second TWS earphone.

10. The human-computer interaction control device for TWS earphones according to any one of claims 7 to 9, characterized in that, The processing unit is also used for: Before recognizing the interaction function corresponding to the detected gesture, it is determined that the TWS earphones are in a wearing state.

11. The human-computer interaction control device for TWS earphones according to claim 7, characterized in that, The processing unit identifies the interaction function corresponding to the detected gesture in the following manner: Based on the predefined relationship between gestures and interactive functions, the interactive function corresponding to the detected gesture is determined; The interactive functions include one or a combination of the following: Volume control function for TWS earphones and / or devices playing media files; The device has control functions for playing the previous or next media file; The device has controls for forward and backward playback of media files. The device has a photo-taking control function.

12. The human-computer interaction control device for TWS earphones according to claim 7, characterized in that, The human-computer interaction control device of the TWS earphone also includes a transmitting unit. The processing unit is further configured to: determine, based on the interaction function type, that the device connected to the TWS earphones will perform the interaction function; The transmitting unit is used to send the control command to the device to which the TWS earphone is connected.

13. A human-computer interaction control device for true wireless TWS earphones, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the human-computer interaction control method for TWS earphones according to any one of claims 1 to 6.

14. A storage medium, characterized in that, The storage medium stores instructions that, when executed by the processor of the TWS earphone, enable the TWS earphone to perform the human-computer interaction control method of the TWS earphone as described in any one of claims 1 to 6.

Images