Wireless audio device and operating method thereof

Abstract

A wireless audio device including two or more earpieces includes: a memory storing one or more computer programs; and one or more processors communicatively connected to one or more microphones, a speaker, and the memory. The one or more computer programs include computer-executable instructions that, when executed individually or collectively by the one or more processors, cause the wireless audio device to measure the magnitude of an ambient audio signal obtained through the one or more microphones, determine a score, indicating the degree to which the ambient audio signal corresponds to a dangerous audio signal, using a dangerous audio signal detection model, and increase the output volume of the ambient audio signal on the basis of the measured magnitude of the ambient audio signal and the determined score.

Description

Wireless audio device and method of operation thereof

[0001] The present disclosure relates to a wireless audio device and a method of operating the same.

[0002] Wireless audio devices can wirelessly connect to electronic devices, such as mobile phones, and output audio data received from the mobile phones. Wireless audio devices can offer users a transparency mode and an active noise cancelling (ANC) mode. A transparency mode allows the user to hear external sounds. ANC mode blocks external noise from the wireless audio device.

[0003] The above information is provided solely as background information to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above information constitutes prior art related to the present disclosure.

[0004] Aspects of the present disclosure may address at least the problems and / or disadvantages mentioned above and provide at least the advantages described below. Accordingly, one aspect of the present disclosure may provide a wireless audio device and a method of operating the same.

[0005] Additional aspects are partly explained in the description that follows, or may be made clear through the description therein or may be understood by practice of the embodiments set forth.

[0006] According to one aspect of the present disclosure, a wireless audio device including two or more earpieces may be provided. Each of the earpieces may include one or more microphones for acquiring ambient audio signals from an external source of the wireless audio device, a speaker for outputting at least one of the acquired ambient audio signals or audio content, a memory having one or more computer programs stored therein, and one or more processors communicatively connected to the one or more microphones, the speaker, and the memory, wherein the one or more computer programs include computer-executable instructions, and when the computer-executable instructions are individually or collectively executed by one or more processors of the wireless audio device, the wireless audio device may measure the magnitude of an ambient audio signal acquired through the one or more microphones, determine a score indicating the degree to which the ambient audio signal corresponds to a hazardous audio signal using a hazardous audio signal detection model, and increase an output volume of the ambient audio signal based on the measured magnitude of the ambient audio signal and the determined score.

[0007] According to another aspect of the present disclosure, a method of operating a wireless audio device may be provided. The method of operating may include: measuring the magnitude of an ambient audio signal acquired through one or more microphones; determining a score indicating the degree to which the ambient audio signal corresponds to a hazardous audio signal using a hazardous audio signal detection model; and increasing the output volume of the ambient audio signal based on the measured magnitude of the ambient audio signal and the determined score.

[0008] According to another aspect of the present disclosure, one or more non-transitory computer-readable storage media may be provided. The one or more non-transitory computer-readable storage media may include one or more computer programs including computer-executable instructions, which, when individually or collectively executed by one or more processors of the wireless audio device, may cause the wireless audio device to perform operations. The operations may include measuring the magnitude of an ambient audio signal acquired through one or more microphones, determining a score indicating the degree to which the ambient audio signal corresponds to a hazardous audio signal using a hazardous audio signal detection model, and increasing the output volume of the ambient audio signal based on the measured magnitude of the ambient audio signal and the determined score.

[0009] Other aspects, advantages and key features of the present disclosure will become apparent to those skilled in the art from the following detailed description of various embodiments of the present disclosure when considered in conjunction with the accompanying drawings.

[0010] Other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent when considered in conjunction with the accompanying drawings, in which:

[0011] FIG. 1 is a block diagram illustrating a wireless audio device and an electronic device according to one embodiment of the present disclosure.

[0012] FIG. 2 is a drawing for explaining a communication environment of a wireless audio device and an electronic device according to one embodiment of the present disclosure.

[0013] FIG. 3 is a drawing illustrating a front view and a back view of a wireless audio device according to one embodiment of the present disclosure.

[0014] FIG. 4 is a diagram for explaining adjusting the playback volume of audio content and the output volume of an ambient audio signal according to one embodiment of the present disclosure.

[0015] FIG. 5 is a diagram for explaining how a reference value determination module determines a reference value according to one embodiment of the present disclosure.

[0016] FIG. 6 is a diagram for explaining adjusting the playback volume of audio content and the output volume of an ambient audio signal according to a reference value according to one embodiment of the present disclosure.

[0017] FIG. 7 is a diagram for explaining adjusting the playback volume of audio content and the output volume of an ambient audio signal according to a reference value according to one embodiment of the present disclosure.

[0018] FIGS. 8A, 8B, and 8C are flowcharts illustrating operations of an earpiece operating method according to various embodiments of the present disclosure.

[0019] FIG. 9 is a flowchart illustrating operations for adjusting the playback volume of audio content and / or the output volume of an ambient audio signal according to a reference value according to one embodiment of the present disclosure.

[0020] It should be noted that the same reference numerals may be used in FIGS. 1 through 9 to indicate identical or similar components, features and structures.

[0021] The following description, with reference to the attached drawings, may be provided to facilitate a comprehensive understanding of various embodiments of the disclosure defined by the claims or their equivalents. While various specific details are included to facilitate understanding, they are merely exemplary. Accordingly, those skilled in the art will appreciate that modifications or variations of the various embodiments described herein may occur without departing from the spirit and scope of the present disclosure. Furthermore, descriptions of well-known functions or components may be omitted for clarity and brevity.

[0022] The words or terms used in the following description are not limited to their dictionary meanings and may be used solely by the inventors to ensure a clear and consistent understanding of the present disclosure. Therefore, those skilled in the art will readily understand that the following descriptions of various embodiments of the present disclosure are provided for illustrative purposes only and are not intended to limit the present disclosure, which is defined by the appended claims and their equivalents.

[0023] Unless the context clearly dictates otherwise, the singular forms “one” and “said” may include plural references.

[0024] The blocks of each flowchart and the combination of flowcharts can be executed by one or more computer programs containing computer-executable instructions. The one or more computer programs may be stored entirely in a single memory device, or the one or more computer programs, divided into different portions, may be stored in multiple memory devices.

[0025] The operations or functions described in this disclosure may be processed by one processor or a combination of processors. A processor or a combination of processors may represent a circuit that performs processing and may include an application processor (AP) (e.g., a central processing unit (CPU)), a communication processor (CP) (e.g., a modem), a graphical processing unit (GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a wireless-fidelity chip, a Bluetooth chip, a global positioning system (GPS) chip, a near field communication (NFC) chip, a connectivity chip, a sensor controller, a touch controller, a fingerprint sensor controller, a display drive integrated circuit (IC), an audio CODEC chip, a USB controller (a universal serial bus controller (USB) controller), a camera controller, and an image processing integrated circuit (image processing integrated circuit). It may include a processing IC, a microprocessor unit (MPU), a system on chip (SOC), an integrated circuit (IC), etc.

[0026]

[0027] FIG. 1 is a block diagram illustrating a wireless audio device and an electronic device according to one embodiment of the present disclosure.

[0028] Referring to FIG. 1, an electronic device (110) may include a processor (111), a memory (112), a display (113), and / or a communication circuit (114).

[0029] The processor (111) may, for example, execute software to control at least one other component (e.g., a hardware or software component) of the electronic device (110) connected to the processor (111), and may perform various data processing or calculations. According to one embodiment of the present disclosure, as at least a part of the data processing or calculation, the processor (111) may store a command or data received from another component (e.g., a communication circuit (114)) in a volatile memory, process the command or data stored in the volatile memory, and store the resulting data in a non-volatile memory. According to one embodiment of the present disclosure, the processor (111) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. For example, when the electronic device (110) includes a main processor and an auxiliary processor, the auxiliary processor may be configured to use lower power than the main processor or to be specialized for a given function. The auxiliary processor may be implemented separately from the main processor or as part of it.

[0030] The auxiliary processor may control at least a part of functions or states related to at least one component (e.g., a display (113), a sensor module (not shown), or a communication circuit (114)) of the electronic device (110), for example, on behalf of the main processor while the main processor is in an inactive (e.g., sleep) state, or together with the main processor while the main processor is in an active (e.g., application execution) state. According to one embodiment of the present disclosure, the auxiliary processor (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (not shown) or the communication circuit (114)). According to one embodiment of the present disclosure, the auxiliary processor (e.g., a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, in the electronic device (110) itself on which the artificial intelligence model is executed, or may be performed through a separate server. Learning algorithms may include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, for example. An artificial intelligence model may include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0031] The memory (112) may store one or more instructions that, when executed, cause the electronic device (110) or the processor (111) to perform various operations. The memory (112) may store various data used by at least one component of the electronic device (110) (e.g., the processor (111), the communication circuit (114)). The data may include, for example, input data or output data for software and commands related thereto. The memory (112) may include volatile memory or non-volatile memory.

[0032] The display (113) can visually provide information to an external device (e.g., a user) of the electronic device (110). The display (113) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment of the present disclosure, the display (113) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch. The display (113) may output an interface screen for controlling, for example, the settings and / or operation modes of the first earpiece (120) and the second earpiece (130).

[0033] The communication circuit (114) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (110) and an external electronic device (e.g., the first earpiece (120) and / or the second earpiece (130)), and the performance of communication through the established communication channel. The communication circuit (114) may operate independently from the processor (111) (e.g., an application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment of the present disclosure, the communication circuit (114) may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device via a first network (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (e.g., a long-range communication network such as a legacy cellular network, a fifth-generation (5G) network, a next-generation communication network, the Internet, or a computer network (e.g., a local area network or a wide area network)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module may use subscriber information stored in the subscriber identification module (e.g., an international mobile subscriber identity (IMSI)) to identify or authenticate the electronic device (110) within a communication network such as the first network or the second network.

[0034] The wireless communication module can support 5G networks and next-generation communication technologies beyond the 4th generation (4G) network, such as new radio access technology (NR). NR access technology can support high-speed transmission of large amounts of data (eMBB (enhanced mobile broadband)), minimizing terminal power and connecting multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency communications (URLLC (ultra-reliable and low-latency communications)). The wireless communication module can support high-frequency bands (e.g., millimeter wave (mmWave) bands) to achieve high data rates, for example. The wireless communication module may support various technologies for securing performance in high frequency bands, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antennas, analog beam-forming, or large scale antennas. The wireless communication module may support various requirements specified in the electronic device (110), external electronic devices, or network systems. According to one embodiment of the present disclosure, the wireless communication module may support a peak data rate (e.g., 20 Gbps or more) for realizing eMBB, a loss coverage (e.g., 164 dB or less) for realizing mMTC, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for realizing URLLC.

[0035] According to one embodiment of the present disclosure, an electronic device (110) may be connected to a first earpiece (120) and / or a second earpiece (130) of a wireless audio device. The electronic device (110) may be connected to the first earpiece (120) through a first link (140), and the electronic device (110) and the first earpiece (120) may communicate in units of time slots set based on a clock of a primary device of the first link (140). The electronic device (110) may be connected to the second earpiece (130) through a second link (150). For example, the electronic device (110) may establish the second link (150) after being connected to the first earpiece (120). In one embodiment of the present disclosure, the second link (150) may be omitted. The first earpiece (120) may be connected to the second earpiece (130) via a third link (160). The first link (140), the second link (150), and the third link (160) may be, for example, communication connections based on Bluetooth communication, but the scope of the embodiment is not limited thereto. The communication environment between the electronic device (110) and the first earpiece (120) and / or the second earpiece (130) is described in more detail in FIG. 2.

[0036] According to one embodiment of the present disclosure, the first earpiece (120) may include a processor (121), a memory (122), a communication circuit (123), a microphone (124), and a speaker (125).

[0037] The processor (121) may, for example, execute software to control at least one other component (e.g., a hardware or software component) of the first earpiece (120) connected to the processor (121), and may perform various data processing or calculations. According to one embodiment of the present disclosure, as at least a part of the data processing or calculation, the processor (121) may store commands or data received from another component (e.g., a communication circuit (123)) in a volatile memory, process the commands or data stored in the volatile memory, and store result data in a non-volatile memory. According to one embodiment of the present disclosure, the processor (121) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith.

[0038] In one embodiment of the present disclosure, the processor (121) may include one or more processors, and the operations of the first earpiece (120) described in the present disclosure may be performed by one processor or by a combination of multiple processors.

[0039] The processor (121) can acquire ambient audio data of the first earpiece (120) using one or more microphones (124). The ambient audio data of the first earpiece (120) may be audio data input to one or more microphones (124) of the first earpiece (120). For example, when a user listens to audio content transmitted from an electronic device (110) while wearing the first earpiece (120), the ambient audio data may include sounds heard from outside, regardless of the audio content.

[0040] In one embodiment of the present disclosure, the processor (121) can dynamically select or determine at least one microphone among a plurality of microphones (124) for acquiring ambient audio data. The processor (121) can acquire ambient audio data using the plurality of microphones (124). The plurality of microphones (124) may be microphones (124) to which beamforming technology is applied. The processor (121) may be configured to process (e.g., perform noise suppression, noise removal, or echo removal) the acquired ambient audio signal.

[0041] According to one embodiment of the present disclosure, the processor (121) may determine a score indicating the degree to which an ambient audio signal corresponds to a hazardous audio signal using a hazardous audio signal detection model. The processor (121) may increase the output volume of the ambient audio signal based on the magnitude of the ambient audio signal and the determined score. For example, the processor (121) may determine a reference value for volume adjustment using the measured magnitude of the ambient audio signal and the determined score, and may increase the playback volume of audio content played back from the first earpiece (120) and / or the second earpiece (130) or decrease the output volume of the ambient audio signal based on the determined reference value.

[0042] The process of determining the reference value is described in more detail in FIG. 5, and the process of adjusting at least one of the playback volume of audio content and the output volume of the ambient audio signal is described in more detail in FIGS. 6 and 7.

[0043] The speaker (125) can output an audio signal to the outside of the first earpiece (120). For example, the speaker (125) can output audio content received from the electronic device (110) and / or ambient audio signals acquired through the microphone (124).

[0044] The communication circuit (123) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the first earpiece (120) and an external electronic device (e.g., electronic device (110) and / or second earpiece (130)), and the performance of communication through the established communication channel. The communication circuit (123) may operate independently from the processor (121) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment of the present disclosure, the communication circuit (123) may include a wireless communication module (e.g., a Bluetooth communication module).

[0045] The microphone (124) may include one or more microphones. The microphone (124) may be configured to detect an audio signal. For example, the microphone (124) may acquire an ambient audio signal transmitted from outside the first earpiece (120) to the first earpiece (120).

[0046] In one embodiment of the present disclosure, when the first earpiece (120) includes a plurality of microphones (124), each of the plurality of microphones may correspond to a different audio receiving path. For example, when the microphone (124) includes a first microphone (e.g., the first microphone (320) of FIG. 3) and a second microphone (e.g., the second microphone (330) of FIG. 3), an ambient audio signal acquired by the first microphone and an ambient audio signal acquired by the second microphone may be treated as ambient audio signals acquired in different audio channels.

[0047] The memory (122) may store one or more instructions that, when executed, cause the processor (121) to perform various operations of the first earpiece (120). The memory (122) may store various data used by at least one component (e.g., the processor (121)) of the first earpiece (120). The data may include, for example, input data or output data for software and commands related thereto. The memory (122) may include volatile memory or non-volatile memory.

[0048] A second earpiece (130) according to one embodiment of the present disclosure may include a processor (131), a memory (132), a communication circuit (133), a microphone (134), and a speaker (135). The processor (131), the memory (132), the communication circuit (133), the microphone (134), and the speaker (135) of the second earpiece (130) may perform the same functions and / or operations as the processor (121), the memory (122), the communication circuit (123), the microphone (124), and the speaker (125) of the first earpiece (120), respectively. Duplicate descriptions are omitted.

[0049] In one embodiment of the present disclosure, the first earpiece (120) and the second earpiece (130) may be, for example, a set of wireless earphones worn on both ears of the user. In this case, the first earpiece (120) may correspond to a wireless earphone worn on the user's left ear, and the second earpiece (130) may correspond to a wireless earphone worn on the user's right ear.

[0050] In one embodiment of the present disclosure, the first earpiece (120) and the second earpiece (130) may be, for example, wireless headsets worn on both ears of the user. In this case, the first earpiece (120) may correspond to a wireless headset part worn on the user's left ear, and the second earpiece (130) may correspond to a wireless headset part worn on the user's right ear.

[0051] When the electronic device (110) and the first earpiece (120) and / or the second earpiece (130) according to one embodiment are connected via a link (e.g., the first link (140) and / or the second link (150)), the electronic device (110) may display one or more user interfaces (UIs) for controlling the operation mode of the first earpiece (120) and / or the second earpiece (130) via the display (113). The user interfaces may include at least one of an interface for an ambient sound mode, an interface for an active noise cancelling (ANC) mode, and an interface for a variable ambient sound mode. The ambient sound mode (or ambient sound permission mode) according to one embodiment is a mode in which an external audio signal of the first earpiece and / or the second earpiece is transmitted to the user via a microphone. The term ambient sound mode may be replaced with, but is not limited to, an ambient sound permission mode or a transparency mode. According to one embodiment, the ANC mode is a mode that blocks external audio signals of the first earpiece and / or the second earpiece. The term ANC mode may be replaced with, but is not limited to, a noise cancelling mode or an ambient sound control mode. According to one embodiment, the variable ambient sound mode is a mode that adjusts the output volume of an external audio signal and / or the playback volume of audio content depending on the external environment of the first earpiece and / or the second earpiece. The variable ambient sound mode may be provided based on the combined use of the ambient sound mode and the ANC mode. The term variable ambient sound mode may be replaced with, but is not limited to, a variable ambient sound permission mode and a variable noise control mode.

[0052] The user can control the user's ambient noise level by selecting one of the ambient sound mode interface, ANC mode interface, and variable ambient sound mode interface provided by the electronic device (110) depending on the noise level of the user's surrounding environment.

[0053]

[0054] FIG. 2 is a drawing for explaining a communication environment of a wireless audio device and an electronic device according to one embodiment of the present disclosure.

[0055] Referring to FIG. 2, the first earpiece (220) may correspond to the first earpiece (120) of FIG. 1, and the second earpiece (230) may correspond to the second earpiece (130) of FIG. 1. The electronic device (210) may correspond to the electronic device (110) of FIG. 1. The electronic device (210) may include, for example, a user terminal such as a smartphone, a tablet, a desktop computer, or a laptop computer. The first earpiece (220) and the second earpiece (230) may correspond to, but are not limited to, wireless earphones, a wireless headset, earbuds, or a speaker system. For example, the first earpiece (220) and the second earpiece (230) may correspond to various types of devices (e.g., hearing aids or portable audio devices) that receive audio signals and output the received audio signals.

[0056] The electronic device (210) and the first earpiece (220) and / or the second earpiece (230) can perform wireless communication over a short distance according to a Bluetooth network. The Bluetooth network may include, for example, a Bluetooth legacy network or a Bluetooth low energy (BLE) network.

[0057] According to one embodiment of the present disclosure, the electronic device (210) may perform a role of a primary device (e.g., a master device), and the first earpiece (220) and / or the second earpiece (230) may perform a role of a secondary device (e.g., a slave device). The number of devices performing a role of secondary devices is not limited to the example illustrated in FIG. 2. According to one embodiment of the present disclosure, the role of the primary device or the secondary device may be determined in an operation in which a link (e.g., 140, 150, and / or 160) between the devices is created. According to another embodiment of the present disclosure, among the first earpiece (220) and the second earpiece (230), one device (e.g., the first earpiece (220)) may perform a role of a primary device, and the other device (e.g., the second earpiece (230)) may perform a role of a secondary device.

[0058] According to one embodiment of the present disclosure, the electronic device (210) can transmit a data packet including audio content to the first earpiece (220) and / or the second earpiece (230). Not only the electronic device (210), but also at least one of the first earpiece (220) and / or the second earpiece (230) can transmit the data packet to the electronic device (210). For example, when audio content (e.g., music) is played on the electronic device (210), the electronic device (210) can transmit a data packet including audio content through a link created with the first earpiece (220) and / or the second earpiece (230) (e.g., the first link (140) and / or the second link (150)), and at least one of the first earpiece (220) and / or the second earpiece (230) can transmit a data packet including data about an ambient audio signal to the electronic device (210) through the created link.

[0059] According to one embodiment of the present disclosure, the electronic device (210) can create or establish a link with at least one of the first earpiece (220) and / or the second earpiece (230) to transmit a data packet. For example, the electronic device (210) can create a first link (140) with the first earpiece (220) and / or a second link (150) with the second earpiece (230) based on a Bluetooth or BLE protocol. In one embodiment of the present disclosure, the electronic device (210) can communicate with the first earpiece (220) via the first link (140). In this case, for example, the second earpiece (230) can be set to monitor the first link (140). The second earpiece (230) can receive data transmitted by the electronic device (210) through the first link (140) by monitoring the first link (140).

[0060]

[0061] FIG. 3 is a diagram showing a front view and a back view of a wireless audio device according to one embodiment of the present disclosure.

[0062] Referring to FIG. 3, the external structure of the first earpiece (e.g., the first earpiece (220) of FIG. 2) is exemplarily described. For the sake of convenience, redundant descriptions are omitted, but the second earpiece (e.g., the second earpiece (230) of FIG. 2) may also have an external structure substantially identical or similar to that of the first earpiece. The external structure of the second earpiece may, for example, have a shape corresponding to the external structure of the first earpiece.

[0063] In one embodiment of the present disclosure, reference numeral (301) illustrates a front view of a first earpiece. The first earpiece may include a housing (310). The housing (310) may form at least a portion of the exterior of the first earpiece. The first earpiece may include a plurality of microphones (320, 330) disposed on a first side of the housing (310) (e.g., a side facing the outside of the ear when the first earpiece is worn). The first microphone (320) and the second microphone (330) may correspond to one or more microphones (124) described in FIG. 1. The first microphone (320) and the second microphone (330) may be disposed to detect sounds coming from a direction toward the user when the first earpiece is worn on the user's body (e.g., the ear). The first microphone (320) and the second microphone (330) may detect sounds from outside the housing (310). For example, the first microphone (320) and the second microphone (330) can detect sounds generated around the first earpiece. The first microphone (320) can detect ambient audio signals coming from the rear or rearward direction (e.g., toward the head) of the user, and the second microphone (330) can detect ambient audio signals coming from the front or forward direction (e.g., toward the face) of the user.

[0064] Depending on the operating mode of the first earpiece, the ambient sound (ambient audio signal) detected through the first microphone (320) and / or the second microphone (330) may be output by the speaker (340). In one embodiment of the present disclosure, the first microphone (320) and the second microphone (330) may be sound-receiving microphones for a noise-cancelling function (e.g., active noise cancellation (ANC)) of the first earpiece. In addition, the first microphone (320) and the second microphone (330) may be sound-receiving microphones for an ambient sound listening function (e.g., a transparency function or an ambient aware function) of the first earpiece. The first microphone (320) and the second microphone (330) may include various types of microphones, including, for example, an electronic condenser microphone (ECM) and a micro electro mechanical system (MEMS) microphone.

[0065] According to one embodiment of the present disclosure, reference numeral (302) illustrates a rear view of the first earpiece. A speaker (340) may be disposed on a second side of the housing (310) (e.g., a side facing the user when the first earpiece is worn). The speaker (340) may correspond to the speaker (125) of FIG. 1. The speaker (340) may convert an electrical signal into an audio signal. The speaker (340) may output audio to an external portion of the first earpiece (e.g., a user's ear). For example, the speaker (340) may convert an electrical signal into an audible sound for the user to perceive. At least a portion of the speaker (340) may be disposed within the housing (310).

[0066]

[0067] FIG. 4 is a diagram for explaining adjusting the playback volume of audio content and the output volume of an ambient audio signal according to one embodiment of the present disclosure.

[0068] Referring to FIG. 4, each of the earpieces included in the wireless audio device (e.g., the first earpiece (120) and the second earpiece (130) of FIG. 1, and the first earpiece (220) and the second earpiece (230) of FIG. 2) can increase the output volume of the ambient audio signal or decrease the playback volume of the audio content based on the measured magnitude of the ambient audio signal and the determined score. For example, each of the earpieces can determine a reference value using the measured magnitude of the ambient audio signal and the determined score, and can adjust at least one of the playback volume of the audio content played in the wireless audio device and the output volume of the ambient audio signal based on the determined reference value.

[0069] In one embodiment of the present disclosure, each of the earpieces may include a first microphone (410), a second microphone (420), a reference value determination module (430), a noise control module (440), an ambient volume control module (450), an audio content playback volume control module (460), and a speaker (470). The operations of the reference value determination module (430), the noise control module (440), the ambient volume control module (450), and the audio content playback volume control module (460) may be performed / controlled by the processor (121) of the first earpiece (120) and / or the processor (131) of the second earpiece (130) described in FIG. 1.

[0070] The first microphone (410) can acquire an ambient audio signal. The first microphone (410) may correspond to the first microphone (320) of FIG. 3. The ambient audio signal represents an audio signal transmitted from outside the wireless audio device. The ambient audio signal acquired through the first microphone (410) can be transmitted to the ambient volume control module (450), the noise control module (440), and the reference value determination module (430).

[0071] The second microphone (420) can acquire an ambient audio signal. The second microphone (420) can correspond to the second microphone (330) of FIG. 3. The ambient audio signal acquired through the second microphone (420) can be transmitted to the reference value determination module (430).

[0072] The reference value determination module (430) can determine the reference value based on the ambient audio signal transmitted from the first microphone (410) and the ambient audio signal transmitted from the second microphone (420). The reference value determination module (430) can determine the reference value based on the size of the ambient audio signal, a score indicating the degree to which the ambient audio signal corresponds to a dangerous audio signal, and the detection time of the ambient audio signal. The process by which the reference value determination module (430) determines the reference value is described in more detail in FIG. 5.

[0073] Audio content refers to content including audio data acquired from an electronic device, and the audio content playback volume can be adjusted through volume control by an audio content playback volume control module (460) of the earpiece or volume control by control of an electronic device (e.g., the electronic device (110) of FIG. 1). For example, the audio content playback volume can be reduced based on the size of an ambient audio signal measured by the earpiece and a determined score. The audio content playback volume control module (460) can adjust the playback volume of audio content played on a wireless audio device based on a reference value. For example, the audio content playback volume control module (460) can reduce the playback volume of audio content played on a wireless audio device based on a gain for the audio content playback volume and a gain corresponding to a determined reference value.

[0074] According to one embodiment of the present disclosure, the playback volume of audio content and the output volume of an ambient audio signal can be adjusted by the control of an electronic device (e.g., the electronic device (110) of FIG. 1). For example, an earpiece (e.g., the first earpiece (120) of FIG. 1 and / or the second earpiece (130) of FIG. 1) can acquire an ambient audio signal and transmit it to the electronic device. The electronic device can determine a score indicating the degree to which the transmitted ambient audio signal corresponds to a dangerous audio signal. If the score is equal to or greater than a score threshold or greater than a first threshold and equal to or less than a second threshold, the electronic device can measure the detection time of the ambient audio signal. The electronic device can measure the magnitude of the ambient audio signal in parallel with determining the score. The electronic device can selectively determine a directional feature value for the ambient audio signal. The electronic device can determine a reference value for adjusting the playback volume of the audio content and the output volume of the ambient audio signal based on the determined magnitude of the ambient audio signal, the score indicating the degree to which the ambient audio signal corresponds to a dangerous audio signal, and the determined directional feature value. When audio content is not being played, the electronic device can transmit a control signal to each earpiece for adjusting the output volume of the ambient audio signal to an output volume of the ambient audio signal corresponding to a determined reference value. When audio content is being played, the electronic device can transmit a control signal to each earpiece for adjusting the output volume of the ambient audio signal and the playback volume of the audio content to an output volume of the ambient audio signal and the playback volume of the audio content corresponding to the determined reference value.

[0075] The ambient volume control module (450) can adjust the output volume of the ambient audio signal. The ambient volume control module (450) can adjust the output volume of the ambient audio signal acquired through the microphone using the measured size of the ambient audio signal and the determined score. For example, the ambient volume control module (450) can reduce the output volume of the ambient audio signal by reducing the gain for the output volume of the ambient audio signal to a gain determined based on the measured size of the ambient audio signal and the determined score.

[0076] The noise control module (440) can control noise through active noise cancellation (ANC). The noise control module (440) can control noise so that the output volume of the ambient audio signal increases. For example, the noise control module (440) can increase the output volume of the ambient audio signal by generating an inverse audio signal having an inverse phase with respect to the ambient audio signal and reducing the gain of the inverse audio signal. Noise can be controlled.

[0077] The speaker (470) can output audio content and ambient audio signals with volume adjusted based on a reference value.

[0078] The wireless audio device may provide an ambient sound listening mode in which external sounds of the wireless audio device input through a microphone are transmitted to the user through a speaker, an ANC mode in which external sounds of the wireless audio device are blocked out, and an adaptive ambient sound mode in which the degree of external sounds blocked out of the wireless audio device is adjusted based on changes in the noise level of the user's surroundings by mixing the ambient sound listening mode and the ANC mode. The adaptive ambient sound mode may allow the wireless audio device to naturally notify the user of hazardous audio signals around the user without disturbing the user.

[0079] When a user wears a wireless audio device and listens to audio content, there is a risk that the user may not clearly perceive hazardous sounds (e.g., a car horn) occurring in the user's surroundings in addition to the audio content, because the wireless audio device seals the user's ear canal. In one embodiment, a wireless audio device may adjust at least one of the playback volume of the audio content and the output volume of an ambient audio signal based on a reference value, thereby informing the user of the occurrence of an external hazardous audio signal, thereby allowing the user to prepare for the danger.

[0080]

[0081] FIG. 5 is a diagram for explaining how a reference value determination module determines a reference value according to one embodiment of the present disclosure.

[0082] The wireless audio device may adjust the output volume of the ambient audio signal (e.g., increase the output volume of the ambient audio signal) or adjust the playback volume of the audio content being played (e.g., decrease the volume of the audio content) based on the measured magnitude of the ambient audio signal, the determined score, the detection time at which the ambient audio signal corresponding to the score is detected, and / or the directional feature value for the ambient audio signal. For example, the wireless audio device may determine a reference value using the magnitude of the ambient audio signal, the determined score, the detection time, and / or the directional feature value, and may adjust at least one of the playback volume of the audio content or the output volume of the ambient audio signal based on the determined reference value. The reference value may be determined by the reference value determination module (430).

[0083] Referring to FIG. 5, the reference value determination module (430) may perform an operation (550) of determining a reference value based on an ambient audio signal acquired through a microphone (e.g., the first microphone (320) and the second microphone (330) of FIG. 3). The reference value represents a reference value used to adjust the volume played or output from a wireless audio device (e.g., the first earpiece (120) and the second earpiece (130) of FIG. 1). For example, the reference value may represent a value determined based on an ambient audio signal to adjust at least one of the playback volume of audio content played from the wireless audio device and the output volume of an ambient audio signal. The term 'reference value' may be replaced with the term 'risk level'.

[0084] A reference value determination module (430) according to one embodiment of the present disclosure may determine a reference value based on the magnitude of an ambient audio signal and a score for the ambient audio signal. For example, the reference value determination module may measure the magnitude of an ambient audio signal acquired through a first microphone (e.g., the first microphone (320) of FIG. 3), determine a score for the ambient audio signal, and then determine a reference value using the measured magnitude of the ambient audio signal and the determined score.

[0085] The reference value determination module (430) according to one embodiment of the present disclosure may determine the reference value based on the magnitude of the ambient audio signal, the score for the ambient audio signal, and the detection time at which the ambient audio signal was measured. For example, the reference value determination module (430) may measure the magnitude of the ambient audio signal acquired through a first microphone (e.g., the first microphone (320) of FIG. 3) and determine a score for the ambient audio signal. The reference value determination module (430) may measure the detection time for the ambient audio signal when the determined score is greater than a first threshold and less than or equal to a second threshold. The reference value determination module (430) may determine the reference value using the magnitude of the measured ambient audio signal, the determined score for the ambient audio signal, and the measured detection time.

[0086] A reference value determination module (430) according to one embodiment of the present disclosure may determine a reference value based on a magnitude of an ambient audio signal, a score for the ambient audio signal, and a directional feature value for the ambient audio signal. For example, the reference value determination module (430) may measure a magnitude for an ambient audio signal acquired through a first microphone (e.g., the first microphone (320) of FIG. 3) and determine a score for the ambient audio signal. The reference value determination module (430) may determine a directional feature value for the ambient audio signal based on the ambient audio signal acquired through the first microphone and the ambient audio signal acquired through the second microphone. The reference value determination module (430) may determine a reference value using the magnitude for the measured ambient audio signal, the determined score for the ambient audio signal, and the determined directional feature value.

[0087] A reference value determination module (430) according to one embodiment may determine a reference value based on a magnitude of an ambient audio signal, a score for the ambient audio signal, a detection time at which the ambient audio signal was measured, and a directional feature value for the ambient audio signal. The magnitude of the ambient audio signal may be measured from the magnitude of the ambient audio signal acquired through a first microphone (e.g., the first microphone (320) of FIG. 3), and the score for the ambient audio signal and the detection time at which the ambient audio signal was measured may be determined based on the ambient audio signal acquired through the first microphone. The directional feature value for the ambient audio signal may be determined based on the ambient audio signal acquired through a second microphone (e.g., the second microphone (330) of FIG. 3).

[0088] In operation (510), the reference value determination module (430) can determine a score representing a probability value that the surrounding audio signal corresponds to a dangerous audio signal.

[0089] The score indicates the degree to which the ambient audio signal corresponds to a hazardous audio signal. The score may be determined as an output value of a neural network that inputs the ambient audio signal as an input. The reference value determination module (430) may input the ambient audio signal to a hazardous audio signal detection model and obtain an output value output by the hazardous audio signal detection model. The hazardous audio signal detection model according to one embodiment may be based on a neural network model. The hazardous audio signal detection model may be a neural network model trained to output a probability value that the input data corresponds to a hazardous audio signal. The hazardous audio signal detection model may include a soft max layer that outputs a probability value that the input data corresponds to a hazardous audio signal. In one embodiment of the present disclosure, the reference value determination module (430) may determine the output value of the soft max layer for the ambient audio signal as a score. In one embodiment, the hazardous audio signal detection model may be a model trained to output a probability value corresponding to a hazardous audio signal using a learning audio signal. For example, a hazard audio signal detection model may be trained using training audio signals that include car horn sound data, vehicle driving sounds, thunder sounds, explosion sounds, ambulance siren sound data, and / or human conversation sound data.

[0090] The reference value determination module (430) can determine whether an ambient audio signal acquired through a microphone is a hazardous audio signal based on a score. The reference value determination module (430) can determine that the ambient audio signal is a hazardous audio signal if the determined score is greater than or equal to a threshold score. For example, the reference value determination module (430) can determine that the ambient audio signal is a hazardous audio signal if the determined score is greater than or equal to 0.5, and can determine that the ambient audio signal is not a hazardous audio signal if the score is less than 0.5. Since a higher score indicates a higher probability that the ambient audio signal is a hazardous audio signal, the reference value determination module (430) can set a higher reference value as the determined score increases.

[0091] In operation (520), the reference value determination module (430) may measure the signal detection time of the ambient audio signal when the determined score is greater than a first threshold value and less than or equal to a second threshold value. According to one embodiment of the present disclosure, the first threshold value may be a value corresponding to 33% of the score percentage, and the second threshold value may be a value corresponding to 66% of the score percentage. For example, when the determined score is greater than the value corresponding to 33% of the score percentage and less than or equal to the value corresponding to 66% of the score percentage, the reference value determination module (430) may measure the signal detection time of the ambient audio signal. The ambient audio signal measured by the reference value determination module (430) may include both continuous audio signals and discrete audio signals. The reference value determination module (430) may measure the signal detection time of the ambient audio signal within a maximum measurable detection time. The reference value determination module (430) may set the reference value to be larger as the measured detection time becomes longer.

[0092] In operation (530), the reference value determination module (430) can measure the size of an ambient audio signal. The size of the ambient audio signal represents the volume level of the ambient audio signal acquired through a microphone (e.g., the first microphone (320) of FIG. 3). The reference value determination module (430) can temporarily or continuously measure the size of the ambient audio signal. The reference value determination module (430) can set the reference value to be larger as the size of the measured ambient audio signal increases within the maximum size limit of the measurable audio signal.

[0093] The directional feature value represents the ratio of the magnitude of the ambient audio signal acquired in a specific direction to the sum of the magnitudes of the ambient audio signals acquired in two or more different directions. For example, the directional feature value may be the ratio of the magnitude of the ambient audio signal acquired from the rear to the sum of the magnitudes of the ambient audio signal acquired in front of the user wearing the wireless audio device and the ambient audio signal acquired from the rear of the user wearing the wireless audio device.

[0094] The directional feature value may be defined by the following mathematical expression 1 or mathematical expression 2, depending on one embodiment.

[0095]

[0096]

[0097]

[0098] In mathematical expressions 1 and 2, back signal power may correspond to the magnitude of the ambient audio signal acquired in the first direction, and back signal power may correspond to the magnitude of the ambient audio signal acquired in the second direction.

[0099] In operation (540), the reference value determination module (430) can optionally determine a directional feature value for the ambient audio signal. The directional feature value can indicate the direction from which the ambient audio signal is heard or the direction to the sound source from which the ambient audio signal is generated. The reference value determination module (430) can obtain the ambient audio signal in a first direction through a first microphone (e.g., the first microphone (320) of FIG. 3) and can obtain the ambient audio signal in a second direction through a second microphone (e.g., the second microphone (330) of FIG. 3). The first microphone and the second microphone can be beamforming microphones. The first microphone can be directed forward with respect to a user wearing the wireless audio device, and the second microphone can be directed backward with respect to the user wearing the wireless audio device. The reference value determination module (430) can determine the directional feature value based on the magnitude of the ambient audio signal obtained by the first microphone and the second microphone.

[0100] The reference value determination module (430) can determine the ratio of the magnitude of the ambient audio signal acquired in the first direction to the sum of the magnitudes of the ambient audio signal acquired in the first direction and the magnitudes of the ambient audio signal acquired in the second direction as defined in mathematical expressions 1 and 2, as a direction feature value.

[0101] According to one embodiment, the reference value determination module (430) may determine a different reference value based on the direction of the ambient audio signal. For example, the reference value determination module (430) may determine a greater directional characteristic value when the acquired ambient audio signal originates from behind the user wearing the wireless audio device than when the acquired ambient audio signal originates from in front of the user.

[0102] According to one embodiment of the present disclosure, the first earpiece and / or the second earpiece can determine the direction of an ambient audio signal (e.g., a hazardous audio signal) (or the direction from which the hazardous audio signal originated). The first earpiece and / or the second earpiece can adjust at least one of a playback volume of audio content and / or an output volume of the ambient audio signal based on the determined direction of the ambient audio signal. For example, if the first earpiece and / or the second earpiece determines that the ambient audio signal is a hazardous audio signal, the first earpiece and / or the second earpiece can increase the output volume of a terrifying audio device closer to the determined direction.

[0103] According to one embodiment of the present disclosure, the first earpiece and / or the second earpiece can determine a direction for an external audio signal based on a determined direction feature value, a magnitude of an external audio signal measured at the first earpiece (e.g., the first earpiece (120) of FIG. 1), and a magnitude of an external audio signal measured at the second earpiece (e.g., the second earpiece (130) of FIG. 1).

[0104] If the direction feature value based on mathematical expression 1 according to one embodiment of the present disclosure is greater than or equal to the threshold direction feature value, the reference value determination module may determine that the direction of the external audio signal (e.g., a dangerous audio signal) is toward the rear of the user wearing the first earpiece. If the determined direction of the external audio signal is toward the rear, the first earpiece may reduce the playback volume of the audio content to the minimum volume and increase the output volume of the ambient audio signal to the maximum volume.

[0105] When the direction feature value based on mathematical expression 1 according to one embodiment of the present disclosure is smaller than the threshold direction feature value, the first earpiece can determine that the direction of the external audio signal is in front of the user wearing the wireless audio device. When the determined direction of the wireless audio signal is in front, the wireless audio device can maintain the playback volume of the audio content and increase the output volume of the ambient audio signal. When the determined direction of the wireless audio signal is in front, the first earpiece can make the ambient audio signal clearly audible without disturbing the user's listening to the audio content by only adjusting the output volume of the ambient audio signal.

[0106] According to one embodiment of the present disclosure, when a difference value between the magnitude of an external audio signal measured by a first earpiece (e.g., a right earpiece) and the magnitude of an external audio signal measured by a second earpiece (e.g., a left earpiece) is greater than or equal to a first threshold magnitude difference value, the first earpiece and the second earpiece may determine that the direction of the external audio signal is toward the right side of a user wearing a wireless audio device. When the direction of the external audio signal is determined to be toward the right side, the second earpiece may maintain the output volume of the ambient audio signal and the playback volume of the audio content, and the first earpiece may maintain the playback volume of the audio content and increase the output volume of the ambient audio signal. When the determined direction of the external audio signal is toward the right side, the first earpiece may clearly transmit the ambient audio signal heard from the right side to the user by only increasing the output volume of the ambient audio signal output from the first earpiece.

[0107] According to one embodiment of the present disclosure, when a difference value between the magnitude of an ambient audio signal measured by a first earpiece (e.g., a right earpiece) and the magnitude of an ambient audio signal measured by a second earpiece (e.g., a left earpiece) is less than or equal to a second threshold magnitude difference value, the first earpiece and the second earpiece may determine that the direction of the external audio signal is toward the left side of a user wearing the wireless audio device. When the direction of the external audio signal is determined to be toward the left side, the first earpiece may maintain the output volume of the ambient audio signal and the playback volume of the audio content, and the second earpiece may maintain the playback volume of the audio content and increase the output volume of the ambient audio signal. When the determined direction of the external audio signal is toward the left side, by increasing only the output volume of the ambient audio signal output from the second earpiece, the second earpiece may clearly transmit the ambient audio signal heard from the left side to the user.

[0108] The above actions (510), (520), (530) and (540) can be performed in parallel or sequentially.

[0109] In operation (550), the reference value determination module (430) may determine the reference value based on all or part of the size of the measured ambient audio signal, the determined score, the measured detection time, and the determined directional feature value. The reference value determination module (430) according to one embodiment of the present disclosure may determine the reference value based on all or part of the determined score, the ratio between the size of the measured ambient audio signal and the set maximum size, the ratio between the measured detection time and the set maximum detection time, and the directional feature value for the ambient audio signal. Typically, a hazard audio signal that indicates danger may be a loud and relatively long audio signal. Therefore, the reference value that may indicate the degree of danger may be proportional to the size of the ambient audio signal and proportional to the duration of the ambient audio signal.

[0110] In the case where the directional feature value according to one embodiment of the present disclosure is a ratio of the magnitude of the ambient audio signal acquired from the rear (or front) to the sum of the magnitudes of the ambient audio signal acquired from the front of a user wearing a wireless audio device and the magnitudes of the ambient audio signal acquired from the rear of the user wearing the wireless audio device, the reference value determination module (430) can set the reference value to be larger as the determined directional feature value is larger (or smaller). In general, since it is difficult for the user to recognize a danger signal generated from the rear of a user wearing a wireless audio device, the reference value can be proportional to the directional feature value.

[0111] The reference value determination module (430) according to one embodiment of the present disclosure determines a score, a percentage value of the sum of the ratio between the size of the measured ambient audio signal and the set maximum size, and a ratio between the measured detection time and the set maximum detection time as a reference value based on Equation 3 or Equation 4, and the reference value can be expressed as a value in the range of 0.0 to 100%. For example, the reference value determination module (430) can determine a score, a percentage value of the sum of the ratio between the size of the measured ambient audio signal and the set maximum size as a reference value according to Equation 3, and optionally, a percentage value of the sum of the ratio between the size of the measured ambient audio signal and the set maximum size, and a ratio between the measured detection time and the set maximum detection time as a reference value according to Equation 4.

[0112]

[0113]

[0114] In Equations 3 and 4, softmax represents a score, time / max time represents the ratio between the measured detection time and the set maximum detection time, and loudness / max loudness represents the ratio between the magnitude of the measured ambient audio signal and the magnitude of the maximum measurable ambient audio signal.

[0115] The reference value determination module (430) according to one embodiment may determine, as the reference value, a percentage value for the product of a score, a ratio between the size of the measured ambient audio signal and the set maximum size, and a ratio between the measured detection time and the set maximum detection time based on Equation 5 or Equation 6. For example, the reference value determination module (430) may determine, as the reference value, a percentage value for the product of a score, a ratio between the size of the measured ambient audio signal and the set maximum size, and a ratio between the measured detection time and the set maximum detection time based on Equation 5, and optionally, may determine, as the reference value, a percentage value for the product of a score, a ratio between the size of the measured ambient audio signal and the set maximum size, and a ratio between the measured detection time and the set maximum detection time based on Equation 6.

[0116]

[0117]

[0118]

[0119] In Equations 5 and 6, softmax represents a score, time / max time represents a ratio between the measured detection time and the set maximum detection time, and loudness / max loudness represents a ratio between the magnitude of the measured ambient audio signal and the magnitude of the maximum measurable ambient audio signal.

[0120] The reference value determination module (430) according to one embodiment may determine a score, a ratio between the size of the measured ambient audio signal and the set maximum size, a ratio between the measured detection time and the set maximum detection time, and a percentage value for the product of the directional feature value for the ambient audio signal as the reference value based on Equation 7 or Equation 8. For example, the reference value determination module (430) may determine a score, a ratio between the size of the measured ambient audio signal and the set maximum size, and a percentage value for the product of the directional feature value as the reference value based on Equation 7, and optionally, may determine a score, a ratio between the size of the measured ambient audio signal and the set maximum size, a ratio between the measured detection time and the set maximum detection time, and a percentage value for the product of the directional feature value as the reference value based on Equation 8.

[0121]

[0122]

[0123] In Equations 7 and 8, softmax represents a score, time / max time represents a ratio between the measured detection time and the set maximum detection time, loudness / max loudness represents a ratio between the magnitude of the measured ambient audio signal and the magnitude of the maximum measurable ambient audio signal, and BF ratio represents a directional feature value for the ambient audio signal.

[0124] A reference value determination module (430) according to one embodiment may optimize a reference value by applying weights to the size of the measured ambient audio signal, the determined score, the measured detection time, and the determined directional feature value. The weights for optimizing each of the determined score, the measured detection time, the size of the measured ambient audio signal, and the determined directional feature value may be represented as w1, w2, w3, and w4.

[0125]

[0126]

[0127]

[0128]

[0129]

[0130] Mathematical expressions 9, 10, 11, 12 and 13 are weighted expressions of Mathematical expressions 3, 4, 5, 6, 7 and 8.

[0131] In mathematical expression 9, if the weight w1 is 1, the weight w2 is 0, and the w3 is 1, the mathematical expression 3 can be obtained, and if the weight w1 is 1, the weight w2 is 1, and the w3 is 1, the mathematical expression 4 can be obtained. In mathematical expression 10, if the weight w1 is 1, the w3 is 1, the mathematical expression 5 can be obtained. In mathematical expression 11, if the weight w1 is 1, the weight w2 is 1, and the w3 is 1, the mathematical expression 6 can be obtained. In mathematical expression 12, if the weight w1 is 1, the w3 is 1, and the w4 is 1, the mathematical expression 7 can be obtained, and in mathematical expression 13, if the weight w1 is 1, the weight w2 is 1, the w3 is 1, and the w4 is 1, the mathematical expression 8 can be obtained. The values ​​of weights w1, w2, w3, and w4 described above are exemplary and are not limited thereto. According to one embodiment, weights w1, w2, w3, and w4 may be defined as values ​​greater than or equal to 0 and less than or equal to 1 to optimize the reference value determination module (430).

[0132] In Equations 9, 10, 11, 12, and 13, softmax represents a score, loudness / max loudness represents the ratio between the magnitude of the measured ambient audio signal and the magnitude of the maximum measurable ambient audio signal, time / max time represents the ratio between the measured detection time and the set maximum detection time, and BF ratio represents a directional feature value for the ambient audio signal.

[0133]

[0134] FIG. 6 is a diagram for explaining adjusting the playback volume of audio content and the output volume of an ambient audio signal according to a reference value according to one embodiment of the present disclosure.

[0135] Referring to FIG. 6, the earpiece (e.g., the first earpiece (120) or the second earpiece (130) of FIG. 1) can simultaneously adjust the playback volume of audio content and the output volume of an ambient audio signal based on a determined reference value. For example, the first earpiece can increase the output volume of the ambient audio signal and decrease the playback volume of the audio content based on the measured size of the ambient audio signal and the determined score. The earpiece can determine the reference value based on mathematical expressions 3 to 13.

[0136] Reference number (610) indicates the range of sizes that the output volume of the ambient audio signal can have. Reference number (611) indicates the minimum value of the output volume of the ambient audio signal, and reference number (612) indicates the maximum value of the output volume of the ambient audio signal.

[0137] Reference number (620) indicates the range of sizes that the playback volume of audio content can have. Reference number (621) indicates the initial value of the playback volume of audio content, and reference number (622) indicates the maximum value of the playback volume of audio content. Reference number (630) indicates the range of reference value sizes. Reference number (631) indicates the minimum value of the reference value, and reference number (632) indicates the maximum value of the reference value. Reference numbers (633) and (634) indicate reference values.

[0138] According to one embodiment of the present disclosure, the earpiece can adjust the playback volume of the audio content and the output volume of the ambient audio signal by adjusting the gain for the audio content playback volume, the gain for the output volume of the ambient audio signal, and the gain for the signal that is opposite in phase to the noise based on a second reference value (634). When the reference value is the second reference value (634), the earpiece can decrease the playback volume of the audio content and increase the output volume of the ambient audio signal based on the gain for the audio content playback volume, the gain for the output volume of the ambient audio signal, and the gain for the signal that is opposite in phase to the noise, which correspond to the second reference value (634). When the reference value according to one embodiment of the present disclosure is changed from the first reference value (633) to the second reference value (634), the earpiece may increase the output volume of the ambient audio signal from the output volume (613) of the ambient audio signal corresponding to the first reference value (633) to the output volume (614) of the ambient audio signal corresponding to the second reference value (634), and may decrease the playback volume of the audio content from the playback volume (624) of the audio content corresponding to the first reference value (633) to the playback volume (623) of the audio content corresponding to the second reference value (634). Through the above process, the output volume of the ambient audio signal may be increased by adjusting to the first target playback volume compared to before being adjusted to the first target playback volume, and the playback volume of the audio content may be decreased by adjusting to the second target playback volume compared to before being adjusted to the second target playback volume.

[0139] An earpiece according to one embodiment can simultaneously adjust the playback volume of audio content and the output volume of an ambient audio signal based on a reference value. The earpiece can adjust the gain of the audio content playback volume for a fixed gain corresponding to the reference value, the gain for the output volume of the ambient audio signal, and / or the gain for a signal that is out of phase with noise.

[0140] Based on this, the playback volume of audio content and the output volume of ambient audio signals can be adjusted.

[0141] Alternatively, the earpiece may simultaneously adjust the playback volume of the audio content and the output volume of the ambient audio signal based on a gain for the current audio content playback volume relative to the current volume, a gain for the output volume of the ambient audio signal relative to the current volume, and / or a ratio of a signal that is out of phase with noise relative to the current volume.

[0142] According to one embodiment, the earpiece may not adjust the output volume of the ambient audio signal if the gain for the output volume of the ambient audio signal is greater than the gain for the output volume of the ambient audio signal corresponding to a determined reference value. If the gain for the output volume of the ambient audio signal corresponding to the determined reference value is less than the gain for the output volume of the ambient audio signal, the size of the output volume of the ambient audio signal may not be adjusted since it is safer to maintain the size of the output volume of the ambient audio signal.

[0143]

[0144] FIG. 7 is a diagram for explaining adjusting the playback volume of audio content and the output volume of an ambient audio signal according to a reference value according to one embodiment of the present disclosure.

[0145] Referring to FIG. 7, the wireless audio device can adjust the output volume of the ambient audio signal and the playback volume of the audio content based on the determined reference value. For example, if the determined reference value is less than a threshold reference value (e.g., the threshold reference value (725)), the wireless audio device can only increase the volume of the ambient audio signal heard by the user, and if the determined reference value exceeds the threshold reference value (e.g., the threshold reference value (725)), the wireless audio device can increase the volume of the ambient audio signal and decrease the playback volume of the audio content. If the determined reference value is a threshold threshold reference value (e.g., the threshold reference value (725)), the wireless audio device can increase the output volume of the ambient audio signal to the maximum and increase the playback volume of the audio content to the maximum.

[0146] Reference number (710) represents a range (716) of the size that the output volume of the ambient audio signal can have and a range (717) of the size that the playback volume of the audio content can have. Reference number (711) represents a minimum value of the output volume of the ambient audio signal, reference number (715) represents a maximum value of the output volume of the ambient audio signal (or a maximum value of the playback volume of the audio content), and reference number (712) represents a minimum value of the playback volume of the audio content. The playback volume of the audio content (714) is a playback volume of the audio content corresponding to a determined reference value (724), and reference number (713) represents an output volume of the ambient audio signal corresponding to a third reference value (723).

[0147] Reference number (720) indicates the range of sizes that the reference value can have. Reference number (721) indicates the minimum value of the reference value, and reference number (722) indicates the maximum value of the reference value. Reference numbers (723) and (724) indicate the determined reference values, and reference number (725) indicates the critical reference value (725) for the reference value.

[0148] A wireless audio device according to one embodiment may increase the output volume of an ambient audio signal to a first target playback volume and maintain the playback volume of audio content when a reference value according to one embodiment is determined to be lower than or equal to a threshold reference value (725). The first target playback volume is the volume of an ambient audio signal corresponding to a reference value lower than or equal to the threshold reference value (725).

[0149] According to one embodiment of the present disclosure, the output volume of an ambient audio signal may be increased by adjusting to the first target playback volume compared to before being adjusted to the first target playback volume. The first target playback volume corresponding to the reference value may be predefined. For example, the first target playback volume corresponding to the threshold reference value (725) may be defined as a volume corresponding to 100% of the volume of the currently input ambient audio signal, and when the reference value is changed from the third reference value (723) to the threshold reference value (725), the wireless audio device may increase the volume of the output ambient audio signal to the volume of the ambient audio signal input through the microphone.

[0150] According to one embodiment, a wireless audio device may increase the output volume of an ambient audio signal to a second target playback volume and decrease the playback volume of audio content to a third target playback volume corresponding to the determined reference value (724), when the determined reference value is greater than the threshold reference value (725). The third target playback volume is a playback volume (714) of the audio content corresponding to the determined reference value (724). The output volume of the ambient audio signal may correspond to a maximum value (715) of the ambient audio signal corresponding to the determined reference value (724). The second target playback volume may be a maximum value (715) of the ambient audio signal corresponding to the determined reference value (724). The output volume of the ambient audio signal may be increased by adjusting to the second target playback volume compared to before being adjusted to the second target playback volume, and the playback volume of the audio content may be decreased by adjusting to the third target playback volume compared to before being adjusted to the third target playback volume. For example, if there is a change from the third reference value (723) to the determined reference value (724), the output volume (713) of the ambient audio signal corresponding to the third reference value (723) may be increased to the maximum value (715) of the ambient audio signal corresponding to the determined reference value (724), and the playback volume of the audio content may be decreased to the playback volume (714) of the audio content corresponding to the determined reference value (724).

[0151]

[0152] FIG. 8A is a flowchart for explaining operations of an operating method of an earpiece according to one embodiment of the present disclosure.

[0153] Referring to FIG. 8A, in operation (810), an earpiece (e.g., the first earpiece (120) of FIG. 1) can measure the size of an ambient audio signal. The earpiece can measure the size of an ambient audio signal acquired through one or more microphones (e.g., microphones (124, 134) of FIG. 1).

[0154] In operation (820), the earpiece may determine a score using a risk audio signal detection model. In one embodiment of the present disclosure, the risk audio signal detection model may include a softmax layer that outputs a probability value that the input data corresponds to a risk audio signal. The earpiece may determine the output value of the softmax layer for the ambient audio signal as a score.

[0155] In operation (830), the earpiece may increase the output volume of the ambient audio signal based on the magnitude and score of the ambient audio signal. For example, the earpiece may determine a reference value using the magnitude and score of the ambient audio signal, and increase the output volume of the ambient audio signal based on the determined reference value.

[0156] In operation (840), the earpiece may reduce the playback volume of the audio content based on the size and score of the ambient audio signal. For example, the earpiece may determine a reference value using the size and score of the ambient audio signal, and reduce the playback volume of the audio content based on the determined reference value.

[0157] An earpiece according to one embodiment may perform an operation of reducing the playback volume of audio content (operation (840)) and an operation of increasing the output volume of an ambient audio signal (operation (830)) in parallel. For example, the earpiece may determine a reference value using the score obtained in operation (820) and the magnitude of the ambient audio signal measured in operation (810), and may increase the output volume of the ambient audio signal and decrease the playback volume of the audio content based on the determined reference value. When the determined reference value is a maximum value, the earpiece may reduce the playback volume of the audio content to a minimum and increase the output volume of the ambient audio signal to a maximum.

[0158]

[0159] FIG. 8b is a flowchart for explaining operations of an operating method of an earpiece according to one embodiment of the present disclosure.

[0160] Referring to FIG. 8b, operations (810) and (820) are identical to operations (810) and (820) of FIG. 8a, respectively, and any duplicate description will be omitted below.

[0161] If the score is less than the score threshold (e.g., 'no' in operation (821)), the earpiece may stop adjusting the playback volume of the audio content and the output volume of the ambient audio signal. The score threshold may be a value corresponding to 50% of the score percentage. If the score threshold according to one embodiment of the present disclosure is a value corresponding to 50% of the score percentage, and the determined score is less than the score threshold, the earpiece may stop adjusting the playback volume of the audio content and the output volume of the ambient audio signal.

[0162] If the score is greater than or equal to the score threshold (e.g., 'yes' in action (821)), the earpiece may measure the detection time of the ambient audio signal in action (822).

[0163] In operation (830-b), the earpiece may increase the output volume of the ambient audio signal based on the magnitude, score, and detection time of the ambient audio signal. For example, the earpiece may determine a reference value using the magnitude, score, and detection time of the ambient audio signal, and increase the output volume of the ambient audio signal based on the determined reference value.

[0164] In operation (840-b), the earpiece may reduce the playback volume of the audio content based on the magnitude, score, and detection time of the ambient audio signal. For example, the earpiece may determine a reference value using the magnitude, score, and detection time of the ambient audio signal, and reduce the playback volume of the audio content based on the determined reference value.

[0165] An earpiece according to one embodiment can perform an operation of reducing the playback volume of audio content (operation (840-b)) and an operation of increasing the output volume of an ambient audio signal (operation (830-b)) in parallel. The operation of the earpiece performing the operation of reducing the playback volume of audio content and the operation of increasing the output volume of an ambient audio signal in parallel corresponds to operation (840) and operation (830) of FIG. 8A, and therefore, any duplicate description thereof will be omitted below.

[0166]

[0167] FIG. 8c is a flowchart for explaining operations of an operating method of an earpiece according to one embodiment of the present disclosure.

[0168] Referring to FIG. 8c, operations (810) and (820) are identical to operations (810) and (820) of FIG. 8a, respectively, and any duplicate description will be omitted below.

[0169] If the determined score is less than or equal to a first threshold value ("yes" in operation (830-1)), the earpiece may stop adjusting the playback volume of the audio content and the output volume of the ambient audio signal. According to one embodiment of the present disclosure, the first threshold value may be a value corresponding to 33% of the score percentage. If the determined score is less than or equal to the value corresponding to 33% of the score percentage, the earpiece may stop adjusting the playback volume of the audio content and the output volume of the ambient audio signal.

[0170] If the determined score is greater than a first threshold value (e.g., “no” in operation (830-1)) or greater than a second threshold value (e.g., “no” in operation (840-1)), the earpiece may increase the output volume of the ambient audio signal or decrease the playback volume of the audio content based on the magnitude of the ambient audio signal and the score without measuring the detection time according to operation (850-1). The second threshold value according to one embodiment of the present disclosure may be a value greater than the first threshold value, and the second threshold value may be a value corresponding to 66% of the score percentage. If the determined score is greater than the second threshold value (e.g., a value corresponding to 66% of the score percentage), the earpiece may determine a reference value using the magnitude of the ambient audio signal and the score. The earpiece may increase the output volume of the ambient audio signal and decrease the playback volume of the audio content based on the determined reference value. If the score is greater than the second threshold, the ambient audio signal is likely to be a hazardous audio signal, so the wireless audio device can clearly notify the user of the hazardous audio signal by adjusting the playback volume of the audio content to the minimum volume and adjusting the output volume of the ambient audio signal to the maximum volume.

[0171] If the determined score is greater than a first threshold (e.g., "no" in action (830-1)) and less than or equal to a second threshold (e.g., "yes" in action (840-1)), then the earpiece can measure the detection time of the ambient audio honeycomb in action (850-1).

[0172] In operation (860-1), the earpiece can increase the output volume of the ambient audio signal based on the size, score, and detection time of the ambient audio signal, and in operation (870-1), the earpiece can decrease the playback volume of the audio content based on the size, score, and detection time of the ambient audio signal. Operations (860-1) and (870-1) correspond to operations (830-b) and (840-b) of FIG. 8B, and therefore, any redundant description will be omitted below.

[0173]

[0174] FIG. 9 is a flowchart illustrating operations for adjusting the playback volume of audio content and / or the output volume of an ambient audio signal based on a reference value according to one embodiment of the present disclosure. The operations for adjusting the playback volume of audio content and the output volume of an ambient audio signal may be performed by an earpiece (e.g., the first earpiece (120) and / or the second earpiece (130) of FIG. 1 ).

[0175] Referring to FIG. 9, in operation (905), the earpiece can acquire an ambient audio signal. The earpiece can acquire an ambient audio signal acquired through one or more microphones (e.g., microphones (124, 134) of FIG. 1).

[0176] In operation (910), the earpiece can measure the size of an ambient audio signal. For example, the earpiece can measure the size of an ambient audio signal acquired through a first microphone (e.g., a first microphone (320) of FIG. 3) and an ambient audio signal acquired through a second microphone (e.g., a second microphone (330) of FIG. 3).

[0177] In operation (920), the earpiece may determine a directional feature value for an ambient audio signal. The earpiece may optionally determine a directional feature value for the ambient audio signal. For example, the earpiece may not determine a directional feature value for the ambient audio signal, but may only determine a magnitude and a score of the ambient audio signal. Alternatively, the earpiece may determine a directional feature value and a score for the ambient audio signal and measure the magnitude of the ambient audio signal.

[0178] In operation (915), the earpiece may determine a score indicating the degree to which the ambient audio signal corresponds to a hazardous audio signal. The earpiece may determine the score using a hazardous audio signal detection model. If the score is less than or equal to a first threshold (e.g., “yes” in operation (924)), the earpiece may stop adjusting the playback volume of the audio content and the output volume of the ambient audio signal. The first threshold according to one embodiment of the present disclosure may be a value corresponding to 33% of the score percentage. If the score is greater than the first threshold and greater than a second threshold (e.g., a value corresponding to 66% of the score percentage) (e.g., “no” in operation (924) and “no” in operation (925)), the earpiece may increase the output volume of the ambient audio signal or decrease the playback volume of the audio content based on the measured magnitude of the ambient audio signal, the determined score, and optionally the determined directional feature. For example, the earpiece may determine a reference value using the measured magnitude of the ambient audio signal, the determined score, and optionally the determined directional feature value, and may increase the output volume of the ambient audio signal or decrease the playback volume of the audio content based on the determined reference value.

[0179] If the score is greater than a first threshold and less than or equal to a second threshold (e.g., “no” in operation (924) and “yes” in operation (925)), then in operation (930), the earpiece may measure a detection time of the ambient audio signal. The earpiece may increase an output volume of the ambient audio signal or decrease a playback volume of the audio content based on the measured magnitude of the ambient audio signal, the determined score, the optionally determined directional feature, and the detection time. For example, the earpiece may determine a reference value using the measured magnitude of the ambient audio signal, the determined score, the optionally determined directional feature, and the detection time, and may increase an output volume of the ambient audio signal or decrease a playback volume of the audio content based on the determined reference value.

[0180] An earpiece according to one embodiment may determine a reference value. The earpiece may determine the reference value based on a magnitude of a measured ambient audio signal, a determined score, a measured detection time, and a directional feature value for the ambient audio signal. The reference value may be determined based on any one of Equations 9 to 13. For example, when the earpiece determines the reference value based on a magnitude of a measured ambient audio signal, a determined score, and a measured detection time, the reference value may be determined based on Equation 11. When the earpiece determines the reference value based on a magnitude of a measured ambient audio signal, a determined score, a measured detection time, and a directional feature value for the ambient audio signal, the reference value may be determined based on Equation 13. When the earpiece determines the reference value based on a magnitude of a measured ambient audio signal and a determined score, the reference value may be determined based on Equation 10, and when the earpiece determines the reference value based on a magnitude of a measured ambient audio signal, a determined score, and a directional feature value for the ambient audio signal, the reference value may be determined by Equation 12.

[0181] The threshold value can be proportional to the magnitude of the measured ambient audio signal, the determined score, and the measured detection time. If the ambient audio signal is a hazardous audio signal, the score may indicate a higher probability than if the ambient audio signal is not a hazardous audio signal. If the ambient audio signal is a hazardous audio signal, the magnitude of the ambient audio signal may be large and the audio signal may be a relatively long duration, and thus the ambient audio signal may be proportional to the magnitude of the ambient audio signal and the measured detection time.

[0182] In general, a user wearing an earpiece may be more at risk if a hazardous audio signal is heard from behind than if the signal is heard from in front of the user. The earpiece can set a higher threshold value for surrounding audio signals as the directional characteristic value increases. The directional characteristic value is not essential for determining the threshold value and may be set optionally.

[0183] If the audio content is not being played (e.g., 'no' in operation (940)), the earpiece may adjust the output volume of the ambient audio signal in operation (945). For example, if the audio content is not being played (e.g., playback of the audio content has been stopped or terminated), the earpiece may adjust the output volume of the ambient audio signal to a first target playback volume corresponding to the determined reference value without adjusting the playback volume of the audio content. The earpiece may increase the output volume of the ambient audio signal by adjusting it to the first target playback volume compared to before it was adjusted to the first target playback volume.

[0184] When audio content is being played (e.g., 'yes' in operation (940)), the earpiece may perform an operation (955) of decreasing the playback volume of the audio content and an operation (950) of increasing the output volume of the ambient audio signal in parallel. In operation (950), the earpiece may increase the output volume of the ambient audio signal. For example, the earpiece may increase the output volume of the ambient audio signal by increasing the output volume of the ambient audio signal to a first target playback volume.

[0185] In operation (955), the earpiece may reduce the playback volume of the audio content. For example, by reducing the playback volume of the audio content to a second target playback volume, the earpiece may reduce the playback volume of the audio content to a second target playback volume.

[0186] An earpiece according to one embodiment may adjust the output volume of an ambient audio signal or the playback volume of audio content based on a reference value. The earpiece may determine the reference value based on any one of mathematical equations 3 to 13. If the determined reference value is less than a threshold reference value (e.g., the threshold reference value of FIG. 7), the earpiece may increase the output volume of the ambient audio signal to a first target playback volume and maintain the playback volume of the audio content.

[0187] When the determined reference value according to one embodiment of the present disclosure is greater than the threshold reference value, the earpiece may increase the output volume of the ambient audio signal to a second target playback volume corresponding to the determined reference value, and decrease the playback volume of the audio content to a third target playback volume corresponding to the determined reference value. The output volume of the ambient audio signal may be increased by adjusting to the second target playback volume compared to before being adjusted to the second target playback volume, and the playback volume of the audio content may be decreased by adjusting to the third target playback volume compared to before being adjusted to the third target playback volume.

[0188] The earpiece can automatically detect sounds that may threaten the user's safety from ambient audio signals and numerically express the risk level of the detected sounds. Based on the numerical reference value, the earpiece can adjust at least one of the playback volume of audio content and the output volume of ambient audio signals, thereby minimizing user interference and alerting the user to the presence of a hazardous audio signal, thereby enabling the user to prepare for danger.

[0189]

[0190] In one embodiment of the present disclosure, a wireless audio device (120; 130; 220; 230; 301; 302) including two or more earpieces, each of the earpieces includes one or more microphones (124; 134; 320; 330; 410; 420) for acquiring an ambient audio signal from outside of the wireless audio device (120; 130; 220; 230; 301; 302), a speaker (125; 135; 340) for outputting at least one of the acquired ambient audio signal or audio content, a memory (122; 132;) including instructions, and one or more processors (121; 131) for executing the instructions, wherein when the instructions are executed by the one or more processors (121; 131), the instructions are configured to: The device (120; 130; 220; 230; 301; 302) can measure the magnitude of the ambient audio signal acquired through the one or more microphones (124; 134; 320; 330; 410; 420), determine a score indicating the degree to which the ambient audio signal corresponds to a hazardous audio signal using a hazardous audio signal detection model, and increase the output volume of the ambient audio signal based on the magnitude of the measured ambient audio signal and the determined score.

[0191] When the instructions are executed by the one or more processors (121; 131), the instructions may cause the wireless audio device (120; 130; 220; 230; 301; 302) to reduce the playback volume of the audio content based on the measured magnitude of the ambient audio signal and the determined score.

[0192] When the instructions are executed by the one or more processors (121; 131), the instructions can cause the wireless audio device (120; 130; 220; 230; 301; 302) to reduce the gain of an anti-phase audio signal for the ambient audio signal output through the speaker so as to increase the output volume of the ambient audio signal.

[0193] When the instructions are executed by the one or more processors (121; 131), the instructions can cause the wireless audio device (120; 130; 220; 230; 301; 302) to increase the output volume of the ambient audio signal as the size of the measured ambient audio signal increases.

[0194] When the instructions are executed by the one or more processors (121; 131), the instructions may cause the wireless audio device (120; 130; 220; 230; 301; 302) to increase the output volume of the ambient audio signal based on a ratio between the measured magnitude of the ambient audio signal and a set maximum magnitude.

[0195] When the instructions are executed by the one or more processors (121; 131), the instructions cause the wireless audio device (120; 130; 220; 230; 301; 302) to measure a detection time at which the ambient audio signal corresponding to the score is detected when the score is greater than a first threshold and less than or equal to a second threshold, and to increase the output volume of the ambient audio signal as the measured detection time becomes longer.

[0196] When the instructions are executed by the one or more processors (121; 131), the instructions may cause the wireless audio device (120; 130; 220; 230; 301; 302) to adjust at least one of a playback volume of audio content played back by the wireless audio device (120; 130; 220; 230; 301; 302) and an output volume of the ambient audio signal based on the measured magnitude of the ambient audio signal and the determined score, if the score is greater than a second threshold value.

[0197] When the instructions are executed by the one or more processors (121; 131), the instructions may cause the wireless audio device (120; 130; 220; 230; 301; 302) to increase the output volume of the ambient audio signal based on a ratio between the measured detection time and a set maximum detection time.

[0198] The one or more microphones (124; 134; 320; 330; 410; 420) include a first microphone (124) for acquiring the ambient audio signal in a first direction and a second microphone (134) for acquiring the ambient audio signal in a second direction different from the first direction, and when the instructions are executed by the one or more processors (121; 131), the instructions can cause the wireless audio device (120; 130; 220; 230; 301; 302) to determine a directional feature value for the ambient audio signal based on a magnitude of the ambient audio signal acquired in the first direction and a magnitude of the ambient audio signal acquired in the second direction.

[0199] When the instructions are executed by the one or more processors (121; 131), the instructions can cause the wireless audio device (120; 130; 220; 230; 301; 302) to determine a ratio of a magnitude of an ambient audio signal acquired in the first direction to a sum of a magnitude of an ambient audio signal acquired in the first direction and a magnitude of an ambient audio signal acquired in the second direction as a directional feature value for the ambient audio signal.

[0200] When the instructions are executed by the one or more processors (121; 131), the instructions may cause the wireless audio device (120; 130; 220; 230; 301; 302) to increase the output volume of the ambient audio signal based on the measured magnitude of the ambient audio signal, the determined score, and the directional feature value for the ambient audio signal.

[0201] When the instructions are executed by the one or more processors (121; 131), the instructions can cause the wireless audio device (120; 130; 220; 230; 301; 302) to increase the output volume of the ambient audio signal as the directional characteristic value for the ambient audio signal increases.

[0202] When the instructions are executed by the one or more processors (121; 131), the instructions can cause the wireless audio device (120; 130; 220; 230; 301; 302) to adjust the output volume of the ambient audio signal to a first target playback volume so that it is larger than before being adjusted to the first target playback volume, and to cause the playback volume of the audio content to be smaller than before being adjusted to the second target playback volume so that the playback volume of the audio content is adjusted to a second target playback volume.

[0203] When the instructions are executed by the one or more processors (121; 131), the instructions cause the wireless audio device (120; 130; 220; 230; 301; 302) to adjust the output volume of the ambient audio signal to a first target playback volume based on the measured magnitude of the ambient audio signal and the determined score, and to maintain the playback volume of the audio content, and to cause the output volume of the ambient audio signal to be greater than before being adjusted to the first target playback volume by the adjustment to the first target playback volume.

[0204] When the instructions are executed by the one or more processors (121; 131), the instructions cause the wireless audio device (120; 130; 220; 230; 301; 302) to adjust the output volume of the ambient audio signal to a second target playback volume and to adjust the playback volume of the audio content to a third target playback volume based on the measured magnitude of the ambient audio signal and the determined score, and the output volume of the ambient audio signal may be made larger by the adjustment to the second target playback volume than before the adjustment to the second target playback volume, and the playback volume of the audio content may be made smaller by the adjustment to the third target playback volume than before the adjustment to the third target playback volume.

[0205] A method of operating a wireless audio device (120; 130; 220; 230; 301; 302) according to one embodiment of the present disclosure may include an operation of measuring the magnitude of the ambient audio signal acquired through one or more microphones (124; 134; 320; 330; 410; 420), an operation of determining a score indicating the degree to which the ambient audio signal corresponds to a hazardous audio signal using a hazardous audio signal detection model, and an operation of increasing an output volume of the ambient audio signal based on the measured magnitude of the ambient audio signal and the determined score.

[0206] The operating method of the wireless audio device (120; 130; 220; 230; 301; 302) may further include an operation of reducing the playback volume of the audio content based on the measured magnitude of the ambient audio signal and the determined score.

[0207] The method of operating a wireless audio device (120; 130; 220; 230; 301; 302) may further include an operation of measuring a detection time at which the ambient audio signal corresponding to the score is detected when the score is greater than a first threshold and less than or equal to a second threshold, and the operation of increasing the output volume of the ambient audio signal may include an operation of increasing the output volume of the ambient audio signal as the measured detection time becomes longer.

[0208] The method of operating a wireless audio device (120; 130; 220; 230; 301; 302) may further include an operation of determining a directional feature value for the ambient audio signal based on a magnitude of the ambient audio signal acquired in a first direction and a magnitude of the ambient audio signal acquired in the second direction, and the operation of increasing the output volume of the ambient audio signal may include an operation of increasing the output volume of the ambient audio signal as the directional feature value for the ambient audio signal increases based on the magnitude of the measured ambient audio signal, the determined score, and the directional feature value for the ambient audio signal.

[0209]

[0210] An electronic device according to an embodiment disclosed in this document may take various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. The electronic device according to an embodiment of this document is not limited to the aforementioned devices.

[0211] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together with the corresponding phrase among the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish the corresponding component from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as being "coupled" or "connected" to another component (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0212] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment of the present disclosure, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0213] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., memory (112), memory (122), memory (132)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (111), processor (121), processor (131)) of a machine (e.g., electronic device (110), first earpiece (120), second earpiece (130)) may call at least one command among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0214] According to one embodiment of the present disclosure, the method according to one embodiment disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0215] According to various embodiments of the present disclosure, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separately arranged in other components. According to various embodiments of the present disclosure, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. Operations performed by a module, program, or other component according to embodiments of the present disclosure may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0216] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and software applications running on the operating system. Furthermore, the processing device may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0217] Software may include computer programs, codes, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, or computer storage medium or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may also be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.

[0218] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination, and the program commands recorded on the medium may be those specially designed and configured for the embodiments of the present disclosure or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0219] The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments of the present disclosure, and vice versa.

[0220] It will be appreciated that various embodiments of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software, as described in the specification and claims.

[0221] Such software may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium may store one or more computer programs (software modules), and the one or more computer programs may include computer-executable instructions that, when executed by one or more processors of the electronic device, cause the electronic device to perform the methods of the present disclosure.

[0222] Such software may be stored in a storage device, such as a read only memory (ROM), whether or not erasable or rewritable, or in a volatile or non-volatile storage device in the form of memory, such as a random access memory (RAM), memory chips, devices or integrated circuits, or optically or magnetically readable media, such as a compact disk (CD), a digital versatile disc (DVD), a magnetic disk or a magnetic tape. It will be appreciated that the storage device and the storage medium are various embodiments of non-transitory machine-readable storage devices suitable for computer programs or stored computer programs containing instructions that, when executed, implement various embodiments of the present disclosure. Accordingly, various embodiments may provide a program comprising code for implementing a method or apparatus as claimed in any of the claims of this specification, and a non-transitory machine-readable storage medium storing such a program.

[0223] While the present disclosure has been shown and described with reference to various embodiments, it will be apparent to those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. In a wireless audio device (120; 130; 220; 230; 301; 302) including two or more earpieces, Each of the above earpieces, One or more microphones (124;134;320;330;410;420) for acquiring ambient audio signals from outside of the wireless audio device (120;130;220;230;301;302); A speaker (125;135;340) outputting at least one of the acquired ambient audio signals or audio contents; Memory (122;132;) storing one or more computer programs; and One or more processors (121; 131) communicatively connected to the one or more microphones, the speaker and the memory Including, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Measure the size of the ambient audio signal acquired through the one or more microphones (124;134;320;330;410;420), Using a risk audio signal detection model, a score is determined indicating the degree to which the above-mentioned ambient audio signal corresponds to a risk audio signal, Comprising computer-executable instructions for increasing the output volume of the ambient audio signal based on the measured magnitude of the ambient audio signal and the determined score. Wireless audio devices (120;130;220;230;301;302).

2. In paragraph 1, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for reducing the playback volume of the audio content based on the measured magnitude of the ambient audio signal and the determined score. Wireless audio devices (120;130;220;230;301;302).

3. In paragraph 1 or 2, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for reducing the gain of an anti-phase audio signal for the ambient audio signal output through the speaker so as to increase the output volume of the ambient audio signal. Wireless audio devices (120;130;220;230;301;302).

4. In any one of paragraphs 1 to 3, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: The larger the size of the measured ambient audio signal, the more computer-executable instructions are included to increase the output volume of the ambient audio signal. Wireless audio devices (120;130;220;230;301;302).

5. In any one of paragraphs 1 to 4, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for increasing the output volume of the ambient audio signal based on a ratio between the magnitude of the measured ambient audio signal and a set maximum magnitude. Wireless audio devices (120;130;220;230;301;302).

6. In any one of paragraphs 1 to 5, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: If the score is greater than the first threshold and less than or equal to the second threshold, the detection time at which the ambient audio signal corresponding to the score is detected is measured, The longer the measured detection time, the more computer-executable instructions are included to increase the output volume of the ambient audio signal. Wireless audio devices (120;130;220;230;301;302).

7. In paragraph 6, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for adjusting at least one of a playback volume of audio content played back by the wireless audio device (120; 130; 220; 230; 301; 302) and an output volume of the ambient audio signal based on the size of the measured ambient audio signal and the determined score when the score is greater than a second threshold value. Wireless audio devices (120;130;220;230;301;302).

8. In paragraph 6, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for increasing the output volume of the ambient audio signal based on a ratio between the measured detection time and a set maximum detection time. Wireless audio devices (120;130;220;230;301;302).

9. In any one of paragraphs 1 to 8, One or more of the above microphones (124;134;320;330;410;420), A first microphone (124) for acquiring the ambient audio signal in the first direction; and Including a second microphone (134) for obtaining the ambient audio signal in a second direction different from the first direction, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for determining a directional feature value for the ambient audio signal based on a magnitude of the ambient audio signal acquired in the first direction and a magnitude of the ambient audio signal acquired in the second direction. Wireless audio devices (120;130;220;230;301;302).

10. In paragraph 9, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for determining a ratio of the magnitude of the ambient audio signal acquired in the first direction to the sum of the magnitudes of the ambient audio signal acquired in the first direction and the magnitudes of the ambient audio signal acquired in the second direction as a direction feature value for the ambient audio signal. Wireless audio devices (120;130;220;230;301;302).

11. In paragraph 9 or 10, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for increasing the output volume of the ambient audio signal based on the magnitude of the measured ambient audio signal, the determined score, and the directional feature value for the ambient audio signal. Wireless audio devices (120;130;220;230;301;302).

12. In any one of paragraphs 1 to 11, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for adjusting the output volume of the ambient audio signal to a first target playback volume based on the size of the measured ambient audio signal and the determined score, and maintaining the playback volume of the audio content. The output volume of the above ambient audio signal is By adjusting to the above first target playback volume, it becomes louder than before being adjusted to the above first target playback volume. Wireless audio devices (120;130;220;230;301;302).

13. In paragraph 12, The one or more computer programs, when individually or collectively executed by the one or more processors (121;131), cause the wireless audio device (120;130;220;230;301;302) to: Further comprising computer-executable instructions for adjusting the output volume of the ambient audio signal to a second target playback volume based on the measured magnitude of the ambient audio signal and the determined score, and for adjusting the playback volume of the audio content to a third target playback volume. The output volume of the above ambient audio signal is By adjusting to the above second target playback volume, it becomes louder than before being adjusted to the above second target playback volume, The playback volume of the above audio content is: By adjusting to the above third target playback volume, it becomes smaller than before being adjusted to the above third target playback volume. Wireless audio devices (120;130;220;230;301;302).

14. In the operating method of a wireless audio device (120; 130; 220; 230; 301; 302), An operation of measuring the magnitude of said ambient audio signal obtained through one or more microphones (124;134;320;330;410;420); An operation of determining a score indicating the degree to which the ambient audio signal corresponds to a hazardous audio signal using a hazardous audio signal detection model; and An operation of increasing the output volume of the ambient audio signal based on the size of the measured ambient audio signal and the determined score; including, How it works.

15. In paragraph 14, An operation for determining a directional feature value for the ambient audio signal based on the magnitude of the ambient audio signal acquired in the first direction and the magnitude of the ambient audio signal acquired in the second direction. Including more, The action of increasing the output volume of the above peripheral audio signal is: An operation of increasing the output volume of the ambient audio signal as the directional feature value for the ambient audio signal increases based on the size of the measured ambient audio signal, the determined score, and the directional feature value for the ambient audio signal, How it works.

Images