Headphone device with full natural user interface and control method thereof

By designing a fully natural interface in the headset and automatically responding to the sounds of users' interest using the microphone and control unit, the existing noise-cancelling headsets have solved the problem of user response difficulties when processing external environment sounds, achieving a more natural and convenient interaction method .

CN110493681BActive Publication Date: 2024-11-05WUXI ZGMICRO ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910831987.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-08-09
Filing Date
2019-09-04
Publication Date
2024-11-05
Estimated Expiration
2039-09-04

AI Technical Summary

Technical Problem

When existing noise-cancelling headphones process external ambient sounds, it is difficult for users to respond quickly to key sounds, and the intelligent listening mode is not intuitive and ineffective, resulting in users not being able to obtain external ambient information in a timely manner when wearing the headphones.

Method used

Design a headphone device to collect ambient sound through a microphone, use the control unit to automatically enter the environmental interaction mode according to the pre-set sound of user interest, and remind the user through visual, tactile or auditory, providing an interactive mode of the entire natural interface, including active attention Ambient sound level controls at level, vocal monitoring, micro monitoring and unlisted levels.

Benefits of technology

It realizes automatic and natural response to external environment sounds when users wear headphones, improves the convenience of interaction between users and the environment, reduces the burden of actively activating the mode of users, and ensures that users can obtain external information in a timely manner when needed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110493681B_ABST
    Figure CN110493681B_ABST
Patent Text Reader

Abstract

The present invention provides an earphone device and a control method thereof, wherein a control unit captures ambient sound through a microphone, and when a preset sound of interest to a user appears in the ambient sound, the control unit determines whether to enter an ambient interaction mode according to the currently perceived ambient sound level; in the ambient interaction mode, the control unit controls the earphone device to output a corresponding ambient interaction reminder according to the currently perceived ambient sound level; wherein the ambient sound level is at least used to set the current state of the earphone device monitoring ambient sound and the form of the currently outputtable ambient interaction reminder; the ambient interaction reminder is a combination of one or more forms of visual reminder, tactile reminder and auditory reminder. Compared with the prior art, the present invention can select and trigger information interaction between the user and the environment in a fully natural interface manner according to the user's usage needs and preferences, which can improve the convenience of interaction, facilitate user use, and facilitate interaction between the outside world and the user.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the field of earphones, and in particular to an earphone device with a fully natural user interface and a control method thereof. [Background technology]

[0002] Headphones are widely used today. In most cases, people use headphones to make calls and listen to music. Currently, some headphones suppress ambient sounds. Some traditional wired and wireless headphones can physically eliminate ambient sounds by partially blocking the ear canal. In addition, active noise cancellation (ANC) headphones can electronically filter out ambient noise. As a result, headphone users often cannot clearly hear the ambient sounds around them when wearing headphones. As a result, these headphone users are usually unable to quickly respond to critical external sounds, such as a person's inquiries, alarms, horns, etc.

[0003] Noise-cancelling headphones currently on the market try to solve the above problems in two ways. One is the "quick reminder mode", which is activated when the user raises his hand to cover one of the earmuffs of the headset. At this time, the volume of the music being played by the headset is lowered, and the external ambient sound is captured and played. The other is the "smart listening mode", in which the user's activity level (i.e. walking, traveling or waiting) is detected to automatically adjust the ambient sound heard by the user.

[0004] However, the "Quick Reminder Mode" requires the user to decide whether to enter the mode, and users often fail to respond in time to sudden, unnoticed interaction requests. For example, you are listening to music with noise-canceling headphones, and your friend will talk to you by calling your name. Since you did not actively enter the "Quick Reminder Mode", you cannot hear your friend calling your name, and you will not actively enter the "Quick Reminder Mode" later. It can be seen that the user's active activation of the "Quick Reminder Mode" still puts a burden on the headphone user to start this function. The "Smart Listening Mode" changes the level of ambient sound heard based on the activity level. However, based on psychology, humans do not change their auditory attention based solely on activity. Instead, humans naturally pay attention to certain key sounds or words (such as names, sirens, greetings, etc.). For example, when the headphone user is driving in a vehicle, the Smart Listening Mode blocks ambient sounds. However, a taxi driver may ask you questions while you are riding in the car, and with Sony's noise-canceling headphones' "Smart Listening Mode", you will not be able to hear the driver's voice. It can be seen that this mode is neither intuitive nor effective.

[0005] Therefore, it is necessary to propose an improved technical solution to solve the above problems. [Summary of the invention]

[0006] The object of the present invention is to provide an earphone device and a control method thereof, which can select and trigger information interaction between a user and an environment in a fully natural interface manner according to the user's usage needs and preferences.

[0007] In order to achieve the above object, a first aspect of the present invention provides an earphone device, comprising:

[0008] A microphone for collecting ambient sound;

[0009] A loudspeaker for playing audio signals;

[0010] A command interface for receiving external control instructions; and

[0011] A control unit with ambient sound monitoring function;

[0012] The control unit captures the ambient sound through a microphone, and when a preset sound of interest to the user appears in the ambient sound, determines whether to enter the ambient interaction mode according to the currently perceived ambient sound level;

[0013] In the environmental interaction mode, the control unit controls the earphone device to output a corresponding environmental interaction reminder according to the currently perceived environmental sound level;

[0014] The ambient sound level is at least used to set the current state of the earphone device monitoring the ambient sound and the form of the currently outputtable ambient interaction reminder; the ambient interaction reminder is a combination of one or more forms of visual reminder, tactile reminder and auditory reminder.

[0015] Furthermore, the auditory reminder may include one or more of a sound of interest to the user, an ambient sound, and a predetermined prompt sound; the control unit controls the speaker to output the auditory reminder;

[0016] The tactile reminder may include a vibration reminder, the earphone device includes a vibration unit, and the control unit outputs the vibration reminder through the vibration unit;

[0017] The visual reminder may include a message reminder and / or a light reminder, the earphone device includes a communication unit, the control unit generates a reminder message, and sends the reminder message through the communication unit.

[0018] Further, the ambient sound level may include at least two of the following levels:

[0019] Active attention level, indicating that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is at least to play all ambient sounds to the headphone user;

[0020] The human voice monitoring level indicates that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is to play the human voice only to the headphone user;

[0021] Micro-monitoring level means that the current headphone device is monitoring the ambient sound, and the ambient interactive reminder that can be output is to play a predetermined prompt sound, tactile reminder or visual reminder only to the headphone user;

[0022] No monitoring level means that the ambient monitoring function of the current headphone device is turned off.

[0023] Furthermore, a storage unit may be included to store environmental noise characteristic information;

[0024] The control unit determines the current ambient sound level, including:

[0025] When the control unit detects the ambient sound level preset by the user, the preset ambient sound level is used as the current ambient sound level; or

[0026] The control unit detects the ambient sound captured by the microphone based on the ambient noise characteristic information to determine the perceived current ambient condition, and determines the current ambient sound level according to a preset correspondence between the ambient sound level and the ambient condition.

[0027] Furthermore, an ambient sound level indicator for indicating the ambient sound level currently perceived by the earphone device may be included, so that the earphone device indicates the currently perceived ambient sound level through the ambient sound level indicator, so that the outside world can know the monitoring status and reminder mode of the earphone;

[0028] The control unit controls the ambient sound level indicator to indicate the current ambient sound level according to the currently perceived ambient sound level.

[0029] Further, the ambient sound level indicator may be an indicator light, and the indicator light may display different lighting effects according to different ambient sound levels; or,

[0030] The ambient sound level indicator may be a display screen, and the display screen displays different images according to different ambient sound levels.

[0031] Furthermore, when the control unit performs the ambient sound monitoring function, it can perform near-field monitoring or far-field monitoring according to the user's choice;

[0032] During near-field monitoring, the microphone can be used to monitor nearby ambient sounds.

[0033] During far-field monitoring, the microphone can be used to monitor distant ambient sounds.

[0034] Furthermore, when the control unit performs the ambient sound monitoring function, it can perform local privacy protection or remote identification according to the user's choice;

[0035] In the case of local privacy protection, the control unit or the local electronic device detects whether there is a sound of interest to the user in the ambient sound according to a predetermined sound recognition algorithm;

[0036] During remote identification, the control unit transmits the ambient sound captured by the microphone to the remote device through the wired / wireless communication unit to detect whether there is a sound of interest to the user in the ambient sound, and receives the detection result from the remote device.

[0037] As a second aspect of the present invention, the present invention provides a method for controlling an earphone device, comprising:

[0038] Steps to capture ambient sound;

[0039] The step of detecting the ambient sound and, when a preset sound of interest to the user appears, determining whether to enter an ambient interaction mode according to the currently perceived ambient sound level;

[0040] In the environmental interaction mode, a step is performed to control the earphone device to output an environmental interaction reminder according to the currently perceived environmental sound level, wherein the environmental sound level is at least used to set the current state of the earphone device monitoring the environmental sound and the form of the currently outputtable environmental interaction reminder; the environmental interaction reminder is a combination of one or more forms of visual reminder, tactile reminder and auditory reminder.

[0041] Furthermore, the auditory reminder may include one or more of the sounds of interest to the user, ambient sounds, and predetermined prompt sounds;

[0042] The tactile reminder includes a vibration reminder;

[0043] The visual reminder includes a message reminder and / or a light reminder.

[0044] Further, the ambient sound level may include at least two of the following levels:

[0045] Active attention level, indicating that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is at least to play all ambient sounds to the headphone user;

[0046] The human voice monitoring level indicates that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is to play the human voice only to the headphone user;

[0047] Micro-monitoring level means that the current headphone device is monitoring the ambient sound, and the ambient interactive reminder that can be output is to play a predetermined prompt sound, tactile reminder or visual reminder only to the headphone user;

[0048] No monitoring level means that the ambient monitoring function of the current headphone device is turned off.

[0049] Furthermore, the method may include presetting the environmental noise characteristic information.

[0050] In the step of determining whether to enter the environment interaction mode according to the currently perceived ambient sound level,

[0051] When the earphone device detects the ambient sound level preset by the user, the preset ambient sound level is used as the current ambient sound level; or,

[0052] The earphone device detects the ambient sound based on the ambient noise characteristic information to determine the perceived current ambient condition, and determines the current ambient sound level according to a preset corresponding relationship between the ambient sound level and the ambient condition.

[0053] Furthermore, the method may further include the step of using an ambient sound level indicator to indicate the currently perceived ambient sound level.

[0054] Further, the ambient sound level indicator is an indicator light, and the indicator light is controlled to display different lighting effects according to different ambient sound levels; or,

[0055] The ambient sound level indicator is a display screen, and the display screen is controlled to display different images according to different ambient sound levels.

[0056] Further, according to the user's selection, near-field monitoring or far-field monitoring is performed in the step of capturing the ambient sound.

[0057] During near-field monitoring, the microphone can be used to monitor nearby ambient sounds.

[0058] During far-field monitoring, the microphone can be used to monitor distant ambient sounds.

[0059] Further, according to the user's choice, when detecting whether a preset user-interested sound appears in the ambient sound, local privacy protection or remote recognition is performed;

[0060] In the case of local privacy protection, the control unit or the local electronic device detects whether there is a sound of interest to the user in the ambient sound according to a predetermined sound recognition algorithm;

[0061] During remote identification, the control unit transmits the ambient sound captured by the microphone to the remote device through the wired / wireless communication unit to detect whether there is a sound of interest to the user in the ambient sound, and receives the detection result from the remote device.

[0062] Compared with the prior art, the earphone device in the present invention can select and trigger information interaction between the user and the environment in a fully natural interface manner according to the user's usage needs and preferences, providing a good interaction method not only for the user but also for the other party interacting with the user, thereby improving the convenience of interaction and facilitating user use.

Brief Description of the Drawings

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0064] Figure 1 is a structural block diagram of an earphone device in one embodiment of the present invention;

[0065] Figure 2 FIG. 4 is a functional flow chart of a method for controlling an earphone device according to an embodiment of the present invention. [Specific implementation method]

[0066] The specific implementation modes of the present invention will be described in detail below with reference to the accompanying drawings.

[0067] In the present invention, "in one embodiment" in various places does not specifically refer to the same embodiment. They can be the same embodiment or different embodiments.

[0068] Based on the analysis of the prior art, we find that the current technology still forces users to adapt to the limitations of technology and follow the traditions and steps set by technology to use headphones. Therefore, the main purpose of the present invention is to improve the headphone device by using the design concept of a fully natural interface (or zero user interface), and to make the headphone device adapt to the needs and preferences of the user by designing a user interface that is consistent with the existing experience or thinking model of the target user group in the expected use scenario, so that the user can fully focus on the activity to be engaged in in the most comfortable and natural way, and forget the existence of the interface itself.

[0069] Based on the above objectives, the present invention provides an earphone device with a fully natural user interface, which can monitor ambient sounds when people wear the earphone device to listen to music or make calls, and automatically control the earphone to output environmental interaction reminders when a pre-set sound of interest to the user appears in the ambient sound. The user interface is a fully natural way, and the user uses it comfortably and naturally.

[0070] The earphone device may be a known or future wireless earphone such as a Bluetooth earphone, or may be a wired earphone.

[0071] Figure 1 FIG. 1 is a structural block diagram of the earphone device 100 in an embodiment of the present invention. Figure 1 As shown, the earphone device 100 includes an attenuator 110, a microphone 120, a control unit 130, a speaker 150, a command interface 160 and a storage unit 180. The attenuator 110, the microphone 120, the speaker 150, the command interface 160 and the storage unit 180 are all electrically connected to the control unit 130.

[0072] The microphone 120 is used to collect signals mainly including ambient sounds, and of course, it also includes some user voices. The microphone 120 can be a microphone or a microphone array.

[0073] The speaker 150 is used to play audio signals.

[0074] The control unit 130 is used to control each module, and may have one or more functional modes.

[0075] In one embodiment, the earphone device 100 may further include a communication unit 140. The communication unit 140 may include a wireless communication unit, which may be used to establish a wireless communication connection with an external electronic device (such as a smart phone, a tablet computer, a laptop computer, a desktop computer, an in-vehicle communication system, etc.), transmit the audio signal from the control unit 130 to the external electronic device, and receive the audio signal from the external electronic device. The wireless communication unit may be a Bluetooth communication unit or other wireless communication units. In another embodiment, the communication unit may include a wired communication unit, which may be used to establish a wired communication connection with an external electronic device (such as a smart phone, a tablet computer, a laptop computer, a desktop computer, an in-vehicle communication system, etc.), transmit the audio signal from the control unit 130 to the external electronic device, and receive the audio signal from the external electronic device.

[0076] The storage unit 180 may pre-store keywords, sound samples, or characteristic parameters of sound samples related to sounds of interest to the user; may pre-store one or more forms of environmental interaction reminders, such as identifiers corresponding to the forms of environmental interaction reminders; and may pre-store a default form of environmental interaction reminders. The storage unit 180 may also pre-store environmental noise characteristic information. Of course, the storage unit 180 may also be used to store other information related to the operation of the headphone device and user information.

[0077] The command interface 160 is used to receive various command information input by the user, including control commands, status setting commands, etc. Among them, the control command includes a command to notify the control unit 130 to enter a corresponding functional mode. After receiving these commands, the command interface 160 transmits the command to the control unit 130 to enable the control unit 130 to enter the corresponding functional mode. The status setting command includes various information input by the user when setting the status information of the headphone device, so that the headphone device can perform corresponding operations according to the status information input by the user when working.

[0078] The command interface 160 may be a button, which is triggered to send a control instruction to the control unit 130. The button may be multiple or one. The command interface 160 may also be a voice recognition interface. When a user speaks a preset voice control instruction, the voice recognition interface transmits the voice control command to the control unit 130.

[0079] Figure 2 FIG. 2 is a functional flow chart of a method 200 for controlling an earphone device according to an embodiment of the present invention.

[0080] like Figure 2 As shown, the control method 200 enters step 220 after starting, the step of capturing ambient sound. At step 220, the ambient sound can be captured by the microphone 120. The captured ambient sound can be temporarily cached in the storage unit 180.

[0081] At step 230 , it is determined whether a preset sound of interest to the user appears in the captured ambient sound. If yes, the process proceeds to step 240 . If not, the process returns to step 220 to continue capturing the ambient sound using the microphone 120 .

[0082] In a specific embodiment, the preset user-interested sound may be a predetermined keyword or a predetermined sound. The predetermined keyword may be a word that the user is interested in, such as a title related to the user's identity, such as the user's name, nickname, position, title, etc., or a commonly used start-up word during a conversation, such as "Hello", "Excuse me", "Sorry", etc. The predetermined sound may be, for example, an alarm sound, a horn sound, a door closing and opening sound, etc.

[0083] The user's sounds of interest may be preset by the user, or may be partially pre-configured by the headphone device when it leaves the factory, such as various alarms, car horns, station announcements, etc. In one embodiment, when the user presets the sounds by himself, the headphone device obtains the sound samples of interest to the user, extracts the characteristic parameters of the sound samples, and stores them by category. To determine whether the user's sounds of interest appear in the ambient sound, a pattern recognition algorithm is used to detect the ambient sound based on the characteristic parameters of the sound samples.

[0084] In step 240, the environment interaction mode is entered, and the environment interaction reminder is output in the environment interaction mode. Then, step 250 is entered to determine whether to exit the environment interaction mode. If yes, the environment interaction mode is exited and the process returns to step 220, and the microphone 120 continues to capture the environment sound. If no, the process returns to step 240, and the earphone device is controlled to continuously collect the environment sound and output the environment interaction reminder. The continuous output of the environment interaction reminder allows the user to fully receive the reminder in the environment interaction mode and complete the entire interaction process.

[0085] The environment interaction reminder includes at least two different reminder forms. The control unit selects the corresponding form of environment interaction reminder according to the user's selection or the perceived external environment, and controls the earphone device to output

[0086] In one embodiment, the environmental interaction reminder can be an auditory reminder. The auditory reminder can be in various forms, such as environmental sound, that is, the external environmental sound is captured in real time by the microphone 120 and played in real time by the speaker 150, which can help the headset user to timely and comprehensively understand the external environment; another example, a predetermined reminder sound, such as a predetermined beep, predetermined music, ringtone, etc.; or a user's interested sound, which can be a predetermined keyword itself, a predetermined sound itself, or a part or all of the external environmental sound containing the user's interested sound, or a voice dialogue containing the user's interested sound extracted from the external environmental sound. For example, if the captured external environmental sound is "Michael, hello, when are you going to China?", and mixed with music, and the pre-set keyword is "Michael", then the output environmental interaction reminder can be the text or machine-simulated voice corresponding to the three words "Michael", or it can be the voice information "Michael" extracted from the external environmental sound, or it can be a complete external environmental sound containing "Michael".

[0087] In addition, the environmental interaction reminder can also be a tactile reminder, for example, by adding a vibration unit 170 to the earphone device, and controlling the vibration unit 170 through the control unit to make the earphone vibrate to achieve a tactile reminder, or by vibrating electronic devices such as smart phones, tablet computers, smart watches, and bracelets used in conjunction with the earphone device to achieve a tactile reminder. The vibration unit 170 can be implemented by an existing eccentric wheel motor or a linear vibration motor, and can also be implemented by other future technologies that can make the device body vibrate.

[0088] Of course, the environmental interaction reminder can also be a visual reminder, for example, a message can be sent in the form of a message notification to a terminal device such as a smart phone, tablet computer, watch, bracelet, etc. used in conjunction with the headset to achieve a visual reminder. For example, when a user wears headphones to watch a movie on a tablet computer, the control unit 130 of the headset device can generate a reminder message based on the information of the sound of interest to the user detected in step 230, and send it to the tablet computer for display through the communication unit of the headset, thereby reminding the user in the form of a screen message. The visual reminder can also be a light reminder, such as setting a flash screen on the display screen of the headset device, or setting a flashing key light.

[0089] The environmental interaction reminder can be one of the above-mentioned auditory reminder, tactile reminder, visual reminder and other possible forms, but in order to provide users with more natural and comfortable environmental interaction reminders, it is best to have multiple environmental interaction reminders. Users can choose one of them in different environments according to different needs and preferences. It can also be a combination of multiple forms, such as a combination of environmental sound and tactile reminder, or a combination of predetermined environmental interaction sound and tactile reminder, or a combination of predetermined keywords in the sound of user interest and visual reminder, etc., so that richer environmental interaction reminders can be output and more convenient to use. More preferably, the headphone device can also automatically sense the external environment and select the most suitable environmental interaction reminder form according to the environmental conditions.

[0090] Therefore, in one embodiment, a certain environmental interaction reminder can be pre-set as a default reminder form in the earphone device, such as an ambient sound. And during the use of the earphone device, a variety of environmental interaction reminder forms are provided to allow the user to select and combine environmental interaction reminder forms according to their own preferences and needs. In this way, in step 240, it will be detected whether there is a user setting. If there is no user setting, the speaker 150 will be controlled to output the environmental interaction reminder in the default reminder form in the environmental interaction mode. If the user setting is detected, the speaker 150, the vibration unit 170 or the communication unit 140 and other functional components can be controlled according to the user's selection and combination to output the environmental interaction reminder separately or jointly.

[0091] In another embodiment, in order to further adapt the earphone device to the needs of the user and provide the most comfortable and natural reminder form, the earphone device can automatically select the corresponding form of environmental interaction reminder output based on the external environment it perceives. The earphone device can pre-store a variety of environmental noise feature information. In step 240, the control unit detects the environmental sound captured by the microphone based on the environmental noise feature information in the environmental interaction mode, determines the current environmental conditions, and automatically selects which or which environmental interaction reminders to output based on the current environmental conditions according to the correspondence between the pre-set environmental conditions and the environmental interaction reminder form. If it is detected that there are multiple car horns and wheel noises when the car is running in the current environment, it can be determined that the surrounding environment is a road. At this time, at least the car horns and human voices in the environmental sounds can be output as sounds of interest to the user, and all captured environmental sounds can also be output so that the user can pay attention to complex traffic safety conditions. For another example, when it is detected that the current ambient sound has a very low noise volume for a long time, or is similar to the pre-stored library noise characteristic information, it is considered that the external environment is very quiet at this time. When a sudden sound is detected in the ambient sound, it is determined that the sudden sound is a sound of interest to the user, and a gentle prompt sound is output, such as "beep...beep...", or a gentle vibration reminder is output.

[0092] In one embodiment, the environment interaction mode can be automatically exited after a certain period of time. In step 240, once the environment interaction mode is entered, the timer starts, and in step 250, it is detected whether the duration of the environment interaction mode reaches a predetermined time, and when the predetermined time is reached, the environment interaction mode is exited.

[0093] In another embodiment, the exit of the environment interaction mode is determined by the user, such as the user controls the end of the environment interaction mode by sending an exit command to the earphone device. Then, in step 250, the command interface 160 is detected to determine whether the command currently input by the user is an instruction to exit the environment interaction mode, and when it is confirmed that the exit command has been received, the environment interaction mode is exited.

[0094] In the third embodiment, the user's voice characteristic parameters are pre-stored in the earphone device. In the environmental interaction mode, the control unit 130 of the earphone device controls the microphone (which may be the microphone 120, preferably the microphone closest to the user's lips) to collect the sound signal near the user's lips, and detects whether the user is making a sound based on the user's voice characteristic parameters. When the detection determines that the user has not made a sound within a predetermined length of time, the step of exiting the environmental interaction mode in step 250 is executed. Compared with the first two embodiments, this method of exiting the environmental interaction mode is more in line with the design concept of a fully natural user interface. The user listens to the environmental interaction reminder in a natural way, and naturally interacts with the environment (talking with people, paying attention to passing vehicles, paying attention to bus station information, etc.), and naturally returns to the audio-visual environment before the environmental interaction when there is no need to interact with the environment. It is extremely comfortable and convenient to use.

[0095] In a preferred embodiment, the control method 200 of the headphone device of the present invention may further include a step of determining whether to enter the environment interaction mode according to the current environment sound level, and a step of outputting a corresponding environment interaction reminder according to the current environment sound level after entering the environment interaction mode. The environment sound level can be used to set the current state of the headphone device monitoring the environment sound and / or the form of the environment interaction reminder that can be currently output, and can also be used to set other working states or working forms of the headphone device.

[0096] In a preferred embodiment, the ambient sound level is used to set the current state of the earphone device monitoring the ambient sound and the currently outputtable form of the ambient interaction reminder, wherein the ambient sound level includes at least two of the following levels:

[0097] Active attention level, indicating that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is at least to play all ambient sounds to the headphone user;

[0098] The human voice monitoring level indicates that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is to play the human voice only to the headphone user;

[0099] Micro-monitoring level means that the current headphone device is monitoring the ambient sound, and the ambient interactive reminder that can be output is to play only the predetermined prompt sound or tactile reminder or visual reminder to the headphone user;

[0100] No monitoring level means that the ambient monitoring function of the current headphone device is turned off.

[0101] Of course, other levels of ambient sound levels can be set according to specific implementation needs.

[0102] The step of determining the current ambient sound level may be based on a user setting method, or based on a method in which the earphone device perceives the surrounding environment, wherein:

[0103] Specifically, based on the user setting mode, when an ambient sound level preset by the user is detected, the preset ambient sound level is used as the current ambient sound level.

[0104] The method of perceiving the surrounding environment based on the earphone device is specifically to detect the ambient sound captured by the microphone based on the ambient noise characteristic information to determine the perceived current environmental condition, and determine the current environmental sound level according to the correspondence between the preset ambient sound level and the environmental condition.

[0105] In one embodiment, when the environment interaction reminder is output in the environment interaction mode, the playing of the audio feed signal currently required to be played in the speaker 150 (such as the music currently being played in the speaker 150) can be paused so that the environment interaction reminder played in the speaker 150 can be clearly heard by the user. Of course, the audio feed signal currently required to be played in the speaker 150 can also be attenuated by the attenuator 110 and then played synchronously with the environment interaction reminder. The audio feed signal currently required to be played can also be played through one of the left channel speaker and the right channel speaker of the speaker 150, and the environment interaction reminder can be played through the other of the left channel speaker and the right channel speaker of the speaker 150. The audio feed signal can come from an external audio reproduction device (such as a mobile phone, a portable player, other earphone devices, audio equipment, etc.) connected to the earphone device by wire or wireless, or from a microphone on the earphone device 100 that is mainly used to collect user voice information.

[0106] In the present invention, the microphone can selectively process prominent external auditory stimuli on behalf of the headphone user, so that the user can focus on what they are doing without missing any relevant auditory clues. When a pre-set sound of interest to the user appears in the ambient sound, a natural and comfortable environmental interaction reminder can be achieved without the need for user action, thereby improving the convenience of interaction and greatly facilitating user use.

[0107] In an application example, user "Zhang San" wears the earphone device 100 and walks on the road. After connecting the earphone device to a smart phone, he listens to music. At this time, user "Zhang San" is immersed in the music. Suddenly, an acquaintance greets "Zhang San" and says "Zhang San, good morning". Since the earphone device 100 monitors the external environmental sound in real time at this time, when the keyword "Zhang San" is monitored, it automatically enters the environmental interaction mode and plays the environmental sound captured by the microphone. In this way, user "Zhang San" can talk with others normally while wearing the earphone device 100 without making any action. After detecting that the conversation between "Zhang San" and others stops, the environmental interaction mode is automatically exited. There is no need for "Zhang San" to actively intervene in the whole process, which is very convenient to use. At the same time, since the earphone actively selects the environmental sound that "Zhang San" is interested in instead of playing all the environmental sounds, it not only purifies "Zhang San's" listening environment, ensures the sound quality when he enjoys music, but also prevents him from missing any key environmental information. This detection and selection mechanism is completely natural because it imitates human intuitive behavior.

[0108] In another application example, user "Zhang San" is wearing the above-mentioned earphone device 100 while driving. Suddenly, an ambulance is coming and the ambulance siren sounds. Since the earphone device 100 is monitoring the external environment sound in real time, when the predetermined sound of the ambulance siren is monitored, it automatically enters the environment interaction mode and plays the ambulance siren monitored by the microphone, and continues to play the siren until the scheduled playback time is reached or "Zhang San" manually cancels it, so that user "Zhang San" has enough time to notice the siren and take corresponding countermeasures. In this way, user "Zhang San" can respond to external emergencies normally while wearing the earphone device 100 without taking any action, which is very convenient.

[0109] Ambient sound level indicator

[0110] In a preferred embodiment, the earphone device further includes an ambient sound level indicator 190, which can indicate the ambient sound level currently perceived by the earphone device 100. The control unit also determines whether to enter the ambient interaction mode according to the currently perceived ambient sound level, and after entering the ambient interaction mode, outputs a corresponding ambient interaction reminder according to the currently perceived ambient sound level.

[0111] The control unit may obtain the currently perceived ambient sound level based on a user setting method or based on a method in which the earphone device perceives the surrounding environment, wherein:

[0112] Specifically, based on the user setting mode, when an ambient sound level preset by the user is detected, the preset ambient sound level is used as the current ambient sound level.

[0113] The method of perceiving the surrounding environment based on the earphone device is specifically to detect the ambient sound captured by the microphone based on the ambient noise characteristic information to determine the current environmental condition perceived, and determine the current environmental sound level according to the correspondence between the preset ambient sound level and the environmental condition. Depending on the different environmental interaction reminders output in the environmental interaction mode, the environmental sound level may include one or more of the active attention level, the human voice monitoring level, the micro monitoring level and the no monitoring level.

[0114] in:

[0115] Active attention level, indicating that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is at least to play all ambient sounds to the headphone user;

[0116] The human voice monitoring level indicates that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is to play the human voice only to the headphone user;

[0117] Micro-monitoring level means that the current headphone device is monitoring the ambient sound, and the ambient interactive reminder that can be output is to play a predetermined prompt sound, tactile reminder or visual reminder only to the headphone user;

[0118] The no-monitoring level indicates that the ambient monitoring function of the current headphone device has been turned off, so that the user can avoid being disturbed. When the ambient sound level currently perceived by the headphone device is the no-monitoring level, since the ambient monitoring function of the current headphone device has been turned off, the ambient interaction mode will not be entered, and no ambient interaction reminder will be output. In this way, the user can actively block the ambient sound by setting the headphone device to the no-monitoring level, thereby avoiding being disturbed.

[0119] Of course, in specific implementation, other ambient sound levels may be obtained by adding or modifying according to the teachings of the present invention.

[0120] The user can select the ambient sound level of the headphone device according to usage habits and scene requirements, and the headphone device can also automatically select the current ambient sound level for the user according to the current environmental conditions it perceives. Under the design concept of the full natural user interface, the natural and comfortable usage needs of most users will be taken into consideration, and the initial ambient sound level of the headphone device can be set to the active attention level.

[0121] In a preferred embodiment, the ambient sound level indicator 190 may be an indicator light 190. At the active attention level, the indicator light is displayed as a lighting effect, such as green, flashing, etc.; at the non-monitoring level, the indicator light is displayed as another state, such as red, non-flashing, etc. In this way, by observing the indicator light 190, it is possible to know the level of ambient sound that the user wearing the headset can hear and whether a conversation is welcome, which gives the headset device a new social attribute. Of course, the ambient sound level indicator 190 can also be implemented using other specific devices, for example, the ambient sound level indicator is a display screen that displays different images depending on the ambient sound level.

[0122] In one application example, user A is wearing the earphone device 100 in the office, and the indicator light 190 is green. Then, after seeing the color of the indicator light 190, colleagues will know that user A welcomes conversation. At this time, colleagues can take the initiative to greet and communicate with user A. User A can also hear friends' voices while wearing headphones, and can respond in time, and there is no need to manually switch the status of the headphones. The whole process is very natural. In another application example, user A is wearing the earphone device 100 and reading a book. At this time, the indicator light 190 is red. Then, after seeing the color of the indicator light 190, others will know that user A does not want to be disturbed and may not be able to hear his voice. At this time, others can avoid disturbing user A, and user A can also avoid being disturbed by others.

[0123] The setting of the ambient sound level indicator 190 may even change the laws, regulations and usage habits in certain regions. For example, the law in California, USA, currently prohibits the use of headphones in both ears while driving, because people are worried that the driver will not be able to hear the warning sounds of others after wearing headphones in both ears. But paradoxically, wearing headphones in only one ear will obviously greatly affect the stereo quality of the headphones, and also bring discomfort to the user. Therefore, this conflict causes most drivers not to use headphones. For businesses, this undoubtedly reduces the consumer group of headphones. The use of an ambient sound level indicator will change this situation. With a unified ambient sound level indicator, drivers can receive ambient sounds at any time, and the police can also clearly see whether the driver has used the appropriate settings (i.e., they can hear ambient noise). This may make it legal for drivers to use headphones in both ears.

[0124] Nearfield and farfield monitoring

[0125] The current headphone devices are basically near-field monitoring, that is, collecting environmental sounds near the headphones. The reason for not setting up far-field monitoring is that the technology is difficult to implement and the cost is high. In addition, far-field monitoring is generally used to issue voice control commands to home appliances, and headphones do not need to be controlled from a distance. More importantly, technicians always habitually think that users are "immersed" in their own little world when using headphones, and they do not need or have time to take care of environmental information from a distance. This ignores some of people's actual needs. For example, when children wear headphones to enjoy music in their bedroom, they also need to hear their mother's call in the kitchen, or the call of friends outside the house. It can be seen that this still falls into the mindset of technology limiting use.

[0126] In a specific embodiment of the present invention, when the control unit performs the ambient sound monitoring function, it can perform near-field monitoring or far-field monitoring according to the user's choice. Near-field monitoring can be performed by default in the initial state. During near-field monitoring, the microphone can be used to monitor nearby ambient sounds, and during far-field monitoring, the microphone can be used to monitor distant ambient sounds. During far-field monitoring, if someone calls the user wearing headphones at a distance (such as 5 to 10 meters), the headphone device can still monitor the relevant call sound, and during near-field monitoring, if someone calls the user wearing headphones at a distance, the headphone device cannot monitor the relevant call sound.

[0127] As a preferred embodiment, in far-field monitoring, the microphone 120 is a microphone array 120. The control unit 130 controls the microphone array 120 to collect ambient sound, and when a sudden sound is detected in the ambient sound in a certain direction, continuous sound source pickup and recognition are performed through sound source positioning to detect whether there is a sound of interest to the user, so as to determine whether to trigger the headset to enter the ambient interaction mode.

[0128] As another preferred embodiment, during far-field monitoring, the microphone 120 is a microphone array 120. The control unit 130 controls the microphone array 120 to collect ambient sound and obtain multiple ambient sound signals. The control unit 130 detects whether there is a sound of interest to the user in each ambient sound signal, and automatically triggers the headset to enter the ambient interaction mode when there is a sound of interest to the user in at least two ambient sound signals.

[0129] Local privacy mode and remote mode

[0130] In a preferred embodiment, when the control unit performs the ambient sound monitoring function, local privacy protection or remote identification is performed according to the user's choice. Local privacy protection can be performed by default in the initial state. Under local privacy protection, the local electronic device or the control unit can detect whether the ambient sound is of interest to the user according to a predetermined sound recognition algorithm, without the need to connect to the Internet to perform relevant identification work with the help of a remote server, which can minimize the problem of privacy leakage. However, Amazon's Alexa and Google Assistant are both connected to the Internet, and the server performs relevant identification work, which poses a risk of privacy leakage because the relevant information may be easily intercepted by a third party through the Internet. However, in order to provide convenience for users, the earphone device of the present invention is also provided with remote identification. During remote identification, the control unit transmits the ambient sound captured by the microphone to the remote device through a wired / wireless communication unit to detect whether the ambient sound is of interest to the user, and receives the detection result from the remote device.

[0131] Partial playback mode and full playback mode

[0132] In a preferred embodiment, the control unit can play part or all of the ambient sounds including sounds of interest to the user monitored by the microphone through the speaker in the environmental interaction mode according to the user's choice. In the partial playback mode, only part of the sound information including sounds of interest to the user such as predetermined keywords or predetermined sounds in the ambient sounds monitored by the microphone is played through the speaker. In the full playback mode, all conversations in the ambient sounds monitored by the microphone can be played. For example: a friend of the headphone user says: "Hello, John. How are you doing lately?" In the partial playback mode, the headphone user can only hear the keyword "Hello, John". In the full playback mode, the headphone user will be able to hear the entire conversation "Hello, John. How are you doing lately?"

[0133] The above description has fully disclosed the specific embodiments of the present invention. It should be pointed out that any changes made by those skilled in the art to the specific embodiments of the present invention do not deviate from the scope of the claims of the present invention. Accordingly, the scope of the claims of the present invention is not limited to the above specific embodiments.

Claims

1. An earphone device, It is characterized in that It includes: A microphone for collecting ambient sound; A loudspeaker for playing audio signals; A command interface for receiving external control instructions; and A control unit with ambient sound monitoring function; The control unit captures the ambient sound through a microphone, and when a preset sound of interest to the user appears in the ambient sound, determines whether to enter the ambient interaction mode according to the currently perceived ambient sound level; In the environmental interaction mode, the control unit controls the earphone device to output a corresponding environmental interaction reminder according to the currently perceived environmental sound level; The ambient sound level is used at least to set the current state of the earphone device monitoring the ambient sound and the form of the currently outputtable ambient interaction reminder; the ambient interaction reminder is a combination of one or more forms of visual reminder, tactile reminder and auditory reminder, The headphone device further comprises an ambient sound level indicator for indicating an ambient sound level currently perceived by the headphone device; The control unit controls the ambient sound level indicator to indicate the current ambient sound level according to the currently perceived ambient sound level.

2. The headphone device according to claim 1, It is characterized in that The auditory reminder includes one or more of the sounds of interest to the user, environmental sounds, and predetermined prompt sounds; the control unit controls the speaker to output the auditory reminder; The tactile reminder includes a vibration reminder, the earphone device includes a vibration unit, and the control unit outputs the vibration reminder through the vibration unit; The visual reminder includes a message reminder and / or a light reminder, the earphone device includes a communication unit, the control unit generates a reminder message, and sends the reminder message through the communication unit.

3. The headphone device according to claim 1, It is characterized in that The ambient sound level includes at least two of the following levels: Active attention level, indicating that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is at least to play all ambient sounds to the headphone user; The human voice monitoring level indicates that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is to play the human voice only to the headphone user; Micro-monitoring level means that the current headphone device is monitoring the ambient sound, and the ambient interactive reminder that can be output is to play a predetermined prompt sound, tactile reminder or visual reminder only to the headphone user; No monitoring level means that the ambient monitoring function of the current headphone device is turned off.

4. The earphone device according to claim 1, 2 or 3, It is characterized in that Also includes a storage unit for storing environmental noise characteristic information; The control unit determines the current ambient sound level, including: When the control unit detects the ambient sound level preset by the user, the preset ambient sound level is used as the current ambient sound level; or, The control unit detects the ambient sound captured by the microphone based on the ambient noise characteristic information to determine the perceived current ambient condition, and determines the current ambient sound level according to a preset correspondence between the ambient sound level and the ambient condition.

5. The headphone device according to claim 1, It is characterized in that The ambient sound level indicator is an indicator light, and the indicator light displays different lighting effects according to different ambient sound levels; or, The ambient sound level indicator is a display screen, and the display screen displays different images according to different ambient sound levels.

6. The headphone device according to claim 1, It is characterized in that When the control unit performs the ambient sound monitoring function, it performs near-field monitoring or far-field monitoring according to the user's selection; During near-field monitoring, the microphone can be used to monitor nearby ambient sounds. During far-field monitoring, the microphone can be used to monitor distant ambient sounds.

7. The headphone device according to claim 1, It is characterized in that When the control unit performs the ambient sound monitoring function, it performs local privacy protection or remote identification according to the user's selection; In the case of local privacy protection, the control unit or the local electronic device detects whether there is a sound of interest to the user in the ambient sound according to a predetermined sound recognition algorithm; During remote identification, the control unit transmits the ambient sound captured by the microphone to the remote device through the wired / wireless communication unit to detect whether there is a sound of interest to the user in the ambient sound, and receives the detection result from the remote device.

8. A method for controlling an earphone device, It is characterized in that It includes: Steps to capture ambient sound; The step of detecting the ambient sound and, when a preset sound of interest to the user appears, determining whether to enter an ambient interaction mode according to the currently perceived ambient sound level; In the environmental interaction mode, a step of controlling the earphone device to output an environmental interaction reminder according to the currently perceived environmental sound level, wherein the environmental sound level is at least used to set the current state of the earphone device monitoring the environmental sound and the form of the currently outputtable environmental interaction reminder; the environmental interaction reminder is a combination of one or more forms of visual reminder, tactile reminder and auditory reminder, It also includes the step of indicating the currently perceived ambient sound level using an ambient sound level indicator.

9. The method according to claim 8, It is characterized in that The auditory reminder includes one or more of the sounds of interest to the user, environmental sounds, and predetermined prompt sounds; The tactile reminder includes a vibration reminder; The visual reminder includes a message reminder and / or a light reminder.

10. The method according to claim 8, Features: The ambient sound level includes at least two of the following levels: Active attention level, indicating that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is at least to play all ambient sounds to the headphone user; The human voice monitoring level indicates that the current headphone device is monitoring the ambient sound, and the ambient interaction reminder that can be output is to play the human voice only to the headphone user; Micro-monitoring level means that the current headphone device is monitoring the ambient sound, and the ambient interactive reminder that can be output is to play a predetermined prompt sound, tactile reminder or visual reminder only to the headphone user; No monitoring level means that the ambient monitoring function of the current headphone device is turned off.

11. The method according to claim 8, 9 or 10, It is characterized in that It also includes the step of presetting the environmental noise characteristic information, In the step of determining whether to enter the environment interaction mode according to the currently perceived ambient sound level, When the earphone device detects the ambient sound level preset by the user, the preset ambient sound level is used as the current ambient sound level; or, The earphone device detects the ambient sound based on the ambient noise characteristic information to determine the perceived current ambient condition, and determines the current ambient sound level according to a preset corresponding relationship between the ambient sound level and the ambient condition.

12. The method according to claim 8, It is characterized in that The ambient sound level indicator is an indicator light, and the indicator light is controlled to display different lighting effects according to different ambient sound levels; or, The ambient sound level indicator is a display screen, and the display screen is controlled to display different images according to different ambient sound levels.

13. The method according to claim 8, Features: According to the user's selection, near-field monitoring or far-field monitoring is performed in the step of capturing environmental sound, When monitoring in the near field, the microphone can be used to monitor the nearby ambient sound. During far-field monitoring, the microphone can be used to monitor distant ambient sounds.

14. The method according to claim 8, Features: According to the user's choice, when detecting whether a preset user-interested sound appears in the ambient sound, local privacy protection or remote recognition is performed; In the case of local privacy protection, the control unit or the local electronic device detects whether there is a sound of interest to the user in the ambient sound according to a predetermined sound recognition algorithm; During remote identification, the control unit transmits the ambient sound captured by the microphone to the remote device through the wired / wireless communication unit to detect whether there is a sound of interest to the user in the ambient sound, and receives the detection result from the remote device.