Portable system for spatial audio detection with alarm elements for the detection of potentially dangerous events.

Abstract

A system comprises a portable device, a plurality of sound sensors installed in the portable device, an alarm element installed in the portable device, and a controller coupled to the plurality of sound sensors and the alarm element. The controller receives a signal indicating a sound detected by at least one of the plurality of sound sensors, determines whether the sound indicates the presence of a potentially hazardous event, ascertains the direction of the sound based on the received signal indicating the sound, activates the alarm element if the sound indicates the presence of the potentially hazardous event, and does not activate the alarm element if the sound does not indicate the presence of the potentially hazardous event.

Description

PRIORITY

[0001] This application claims priority over the preliminary US patent application No. 63 / 531,861, filed on August 10, 2023, the contents of which are hereby incorporated in full. TECHNICAL AREA

[0002] The present disclosure relates to systems and methods for providing spatial audio detection and haptic feedback to warn of a potentially dangerous event. BACKGROUND

[0003] When a person moves around their environment, they may encounter various dangers and life-threatening situations. They may be warned of these dangers by auditory signals (e.g., sirens, alarms, or verbal warnings). While hearing-impaired individuals can orient themselves fairly well in a room, they have difficulty perceiving dangers in their environment and reacting quickly because they may only be able to perceive such auditory warnings to a limited extent or not at all. If the source of the danger is outside the individual's field of vision, they may have no indication of its presence.

[0004] Therefore, there is a need for improved systems and procedures to provide information about potentially dangerous events in the vicinity of a hearing-impaired person for whom an acoustic warning is provided, in a convenient and effective manner. SUMMARY

[0005] Examples in this disclosure provide systems that deliver spatial audio detection and tactile feedback to the system's users. According to one example, a system includes a portable device, a plurality of sound sensors installed in the portable device, an alarm element installed in the portable device, and a controller coupled to the plurality of sound sensors and the alarm element.The controller receives a signal indicating a sound detected by at least one of the multiple sound sensors, determines whether the sound indicates the presence of a potentially hazardous event, determines the direction of the sound based on the received signal indicating the sound, activates one or more of the alarm elements if the sound indicates the presence of the potentially hazardous event; and does not activate the alarm element if the sound does not indicate the presence of the potentially hazardous event.

[0006] According to another example, a method involves receiving a signal indicating a sound detected by at least one of the plurality of sound sensors; determining whether the sound indicates the presence of a potentially hazardous event; determining the direction of the sound based on the received signal indicating the sound; if the sound indicates the presence of the potentially hazardous event, activating an alarm element in a portable device; and if the sound does not indicate the presence of the potentially hazardous event, not activating the alarm element in the portable device.

[0007] According to another example, a device includes a sound sensor interface; an alarm element interface; and a controller. The controller receives a signal indicating a sound detected by at least one of a plurality of sound sensors in a portable device; determines whether the sound indicates the presence of a potentially hazardous event; determines the direction of the sound based on the received signal indicating the sound; activates an alarm element in the portable device if the sound indicates the presence of the potentially hazardous event; and does not activate the alarm element in the portable device if the sound does not indicate the presence of the potentially hazardous event. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Exemplary aspects of the present revelation are described below in connection with the figures in which: Fig. 1A and Fig. 1B Schematic representations of an example system for providing spatial audio detection and tactile feedback according to examples in the present disclosure are; Fig. 2 a schematic diagram of an exemplary network of sound sensors according to examples in the present disclosure; Fig. 3 a schematic diagram of an exemplary network of alarm elements according to examples in the present disclosure; Fig. 4 is an illustration of an example system for providing spatial audio detection and tactile feedback according to examples in the present disclosure; Fig. 5 a flowchart of the operation of an example system for providing spatial audio detection and tactile feedback according to examples in the present disclosure; and Fig. Figure 6 is an example diagram of the operation of an example system for providing spatial audio detection and tactile feedback according to examples in the present disclosure.

[0009] It is understood that the reference signs for each illustrated element appearing in several different figures have the same meaning in all figures, and that the mention or discussion of an illustrated element in connection with a particular figure also applies to any other figure in which the same illustrated element is shown. DETAILED DESCRIPTION

[0010] Fig. Figure 1A is a schematic representation of an example system 100 for providing spatial audio detection and tactile feedback, according to examples in this disclosure. The system 100 includes a portable device 110, sound sensors 120, one or more alarm elements 130, a device 140, and a controller 150. The system 100 implements a method for detecting potentially hazardous events, for which an acoustic warning is provided, in the vicinity of a hearing-impaired person and alerts the person using alarm elements embedded in a portable device. By using a device that generates 3D mapping of sounds into non-audible warnings (e.g., vibrations and / or visual warnings) when the sounds are considered an indication of a potentially hazardous event, the attention of hearing-impaired persons is increased.System 100 can be implemented using special hardware and software.

[0011] The portable device 110 is a device worn by the user. While it is in Fig. As illustrated in Figure 1A as a cap, the wearable device 110 can be any device worn by the user, such as a bracelet, glasses, ring, vest, headband or any other wearable device that enables spatial directional notification. Fig. Figure 1B illustrates, for example, the wearable device 110 as a vest worn by the user on their upper body. Because the wearable device 110 is worn by the user, it is easy for the user to interact with it. In some embodiments, the wearable device 110 can be a combination of several devices. For example, the wearable device 110 can consist of one device containing sound sensors 120 and another device containing one or more alarm elements 130, with the devices communicating with each other. Another example: The wearable device 110 can be configured as a device containing a subset of sound sensors 120 and one or more alarm elements 130, and another device containing a different subset of sound sensors 120 and one or more alarm elements 130.

[0012] The sound sensors 120 are configured as any type of audio sensor, decibel measuring device, or level sensor. Each sound sensor 120 can, for example, be a microphone with appropriate circuitry. Alternatively, the sound sensors 120 can form a network of microphones, each coupled to a single processing circuit, such as the controller 150. The portable device 110 incorporates at least three sound sensors 120 to form a network of sensors that can be used to identify the direction of sounds. During operation, the sound sensors 120 monitor the user's environment for sounds. The sound sensors 120 are positioned within the portable device 110 to provide 360° coverage around the device. Sound indications detected by the sound sensors 120 are transmitted to the controller 150.Controller 150 analyzes the signals, and if the sounds are considered an indication of a potentially dangerous event, system 100 instructs one or more alarm elements 130 to generate tactile feedback to the user. The placement and operation of the sound sensors 120 are described in [reference to relevant document]. Fig. 2 described in more detail.

[0013] The alarm elements 130 are comprised of all types of devices capable of providing tactile feedback to the user, such as vibration, light, temperature, or any other suitable non-acoustic feedback. The alarm elements 130 are coupled to the controller 150. When the system 100 commands one or more alarm elements 130 to generate tactile feedback, the one or more alarm elements 130 can be activated to warn the user of a potential hazard. For example, in response to the command, the one or more alarm elements 130 can vibrate, light up, and / or change temperature. The alarm elements 130 are placed in the portable device 110 to form a network of stimulating components that generate haptic feedback indicating the presence, direction, and / or intensity of sound.The alarm elements 130 provide the user of the portable device 110 with information about the presence, direction, and / or intensity of sound detected by one or more sound sensors 120. For example, a subset of the alarm elements 130 can vibrate to warn the user. This subset of alarm elements 130 can correspond to the direction of the sound indicating the potentially hazardous event detected by the sound sensors 120. In some embodiments, the level at which the alarm elements 130 vibrate corresponds to the volume of the sound detected by the sound sensors 120. In other embodiments, a subset of the alarm elements 130 can illuminate to indicate the presence of a potentially hazardous event and flash at a level corresponding to the volume of the sound detected by the sound sensors 120.In another embodiment, a subset of the alarm elements 130 can change their temperature (e.g., heat up or cool down) to indicate the presence of a potentially hazardous event, and the magnitude of the temperature change can correspond to the volume of sound detected by the sound sensors 120. The correspondence between the volume of detected sound and the level of one or more alarm elements 130 can be linear, logarithmic, or a combination thereof. The placement and operation of the alarm elements 130 are described in [reference]. Fig. 3 described in more detail.

[0014] The system 100 can include a controller 150 in the operable device 100, which controls the operation of the operable device 110, the sound sensors 120 and / or the alarm elements 130 in conjunction with or independently of the software application running on the device 140, as shown in Fig. 4 described in more detail. In addition to the controller 150, the system 100 can include a device 140, which provides a user interface through which the user can calibrate, configure, and / or control the operation of the portable device 110, the acoustic sensors 120, and / or the alarm elements 130. The device 140 can be any device that has a processor capable of running a software application. For example, the device 140 can be a mobile phone, a tablet computer, a laptop, a personal computer, or a PDA.

[0015] The software application running on the device 140 can communicate with the controller 150 to calibrate the sound sensors 120 and / or the alarm elements 130 and to set the parameters associated with the sound sensors 120 and / or the alarm elements 130, such as the setting of the "danger threshold" and the maximum intensity of the haptic feedback provided by one or more alarm elements 130. In some embodiments, the system 100 also monitors ambient sounds, and the controller 150 analyzes these sounds to determine whether they exceed the "danger threshold" or whether certain keywords indicating a hazard are detected. For example, the controller 150 analyzes the detected sounds and determines, based on certain criteria, whether they are considered an indication of a potentially hazardous event.If this is the case, the controller 150 generates haptic or other feedback by actuating one or more change elements 130 to provide vibrations that indicate the intensity and direction of the sounds exceeding the "danger threshold" or that detect specific keywords indicating a hazard. In other embodiments, the software application running on the device 140 analyzes the sounds to determine whether the sounds exceed the "danger threshold" or whether specific keywords indicating a hazard have been detected, and generates haptic or other feedback by actuating one or more change elements 130 to provide vibrations that indicate the intensity and direction of the sounds exceeding the "danger threshold" or the detected specific keywords indicating a hazard.The functionality of the software application and / or the controller 150, the danger threshold and the keyword recognition are described in the . Fig. 5 and Fig. 6 described in more detail.

[0016] Fig. Figure 2 is a schematic representation of an exemplary network of sound sensors, which includes the portable device 110 according to the examples in the present disclosure. The in Fig. The embodiment shown in Figure 2 includes three sound sensors 220a-220c, but the portable device 110 can also include more than three sound sensors 220.

[0017] The sound sensors 220 are arranged in an equilateral triangular configuration with three sensors. Each sound sensor 220 has a primary contribution for a 120° sector around the portable device 110. If the portable device 110 includes more than three sound sensors 220, each sound sensor 220 has a primary contribution for a smaller sector around the portable device 110. For example, if the portable device 110 includes six sound sensors 220, each sound sensor 220 can provide a primary contribution for sounds in a 60° sector around the portable device 110. As the number of sound sensors 220 increases, a smaller sector is determined for the primary contribution of each sound sensor 220, and the accuracy is improved.

[0018] When detecting sound, a sound sensor 220 that is closest to the sound source will detect the sound at a higher level than sound sensors 220 that are further away from the sound source. Fig. For example, sound sensor 220a is closest to sound source 250 and detects a higher level than sound sensors 220b and 220c. If the sounds detected by a sound sensor 220 exceed a user-configurable sound threshold, the data from all sound sensors 220 are analyzed using a software application and / or a controller, and the software application and / or controller determines the direction of the detected sound. The software application and / or controller can use computational techniques, such as multi-channel compensation and / or keyword recognition, to analyze the detected sounds, determine whether the sounds indicate a potentially hazardous event, and determine the direction of the potential hazard source. These techniques are described in the Fig. 4 and Fig. 5 described in more detail.

[0019] Fig. Figure 3 is a schematic representation of an exemplary network of alarm elements according to the examples in the present disclosure, which are included in the portable device 110. The in Fig. The embodiment shown in Figure 3 includes eight alarm elements 330a-330h, but the portable device 110 can also include more or less than eight alarm elements 330.

[0020] As soon as the system 100 detects the presence of a sound indicating a potential hazard and specifies the direction of the sound's source, the system 100 activates one or more alarm elements 330 to notify the user of the potential hazard. The number of alarm elements 330 activated by the system 100 corresponds to the direction of the sound's source. Fig. For example, if System 100 determines that sound source 350 indicates the presence of a potential hazard, it then determines the direction of sound source 350 and activates alarm elements 330a and 330b, which are closest to sound source 350. System 100 does not activate alarm elements 330c-330h. This allows the user of System 100 to determine that the potential hazard is located in the direction of the activated alarm elements 330a and 330b.

[0021] In some embodiments, the intensity of the activation (e.g., vibration, flashing light, and / or temperature change) of the activated alarm elements 330 corresponds to the volume of the sound source 350. For example, the larger the volume of the sound source 350, the more intense the activation of the activated alarm elements 330. The intensity of the activation of the activated alarm elements 330 can be directly proportional to the volume of the sound source 350 or based on a logarithmic correlation.

[0022] Fig. Figure 4 illustrates an example system for providing spatial audio detection and tactile feedback according to the examples in this disclosure. System 400 includes the sound sensors 420a-420d, which are described in Fig. 1A, Fig. 1B and Fig. The sound sensors 120 and 220a-220c described in the 2 may be similar. System 400 also includes alarm elements 430a-430d, which correspond to the ones described in the Fig. 1A, Fig. 1B and Fig. The sound sensors 130 and 330a-330c described in section 3 may be similar. System 400 further includes device 440, including a software application executed by device 440. Device 440 may be similar to the one described in section 3. Fig. 1A and Fig. Device 140 described in 1B should be similar.

[0023] The System 400 also includes the portable controller 450. The portable controller 450 can be implemented using analog circuitry, digital circuitry, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), reconfigurable logic, programmable logic devices (PLDs), instructions for execution by a processor, or any suitable combination thereof. The portable controller 450 can be coupled with the sound sensors 420a-420d and the alarm elements 430a-430d. In particular, the portable controller 450 can include a sound sensor interface 452 to receive data relating to sounds detected by one or more sound sensors 420a-420d.The portable controller 450 can also include the alarm element interface 454 to send signals to one or more alarm elements 430a-430d to activate one or more alarm elements 430a-430d in response to sound detection.

[0024] The portable controller 450 can control the operation of the sound sensors 420a-420d and the alarm elements 430a-430d by receiving sound measurements from the sound sensors 420a-420d, analyzing the sound measurements, determining the direction of the sound, and generating a signal to activate the alarm elements 430a-430d, as described in relation to Fig. 5 further described.

[0025] The portable controller 450 can communicate with the device 440 via the communication link 460. The communication link 460 can be any suitable wired or wireless communication protocol, such as an internet protocol, a mobile phone protocol, a wireless local area network (WLAN) (e.g., the IEEE 802.11 protocol), or various short-range wireless communication links, such as Bluetooth, NFC, or ZigBee. A user of the portable device, which includes sound sensors 420a-420d and alarm elements 430a-430d, can interact with the device 440 to calibrate, configure, and / or control the system 400, as described in Fig. 5 described in more detail.

[0026] Fig. Figure 5 is a flowchart of the operation of an example system for providing spatial audio detection and tactile feedback, according to examples in this disclosure. Method 500 can be implemented using a portable controller, such as the portable controller 450, a software application executed by the device 140 or the device 440, a suitable combination thereof, or any other system operable to perform Method 500. Although the examples described above, other variations and examples from this disclosure can be made without departing from the spirit and scope of these disclosed examples.

[0027] Procedure 500 begins in step 510, in which procedure 500 initializes a system for providing spatial audio detection and tactile feedback, e.g., system 100 and / or system 400. The initialization may include turning on the system's sound sensors and alarm elements, such as sound sensors 120, 220, and / or 420 and alarm elements 130, 330, and / or 430.

[0028] Procedure 500 proceeds to step 520 to calibrate the system. Calibration may include calibrating the sound sensors and alarm elements, such as mapping the sound sensors to the alarm elements so that the system activates the alarm elements based on the detected sounds. Calibration may also include user-specific adjustments, such as setting the activation intensity, the danger threshold, the sound sensors or levels that trigger an alarm, and the environmental profiles. For example, the user can specify that they will only be alerted by sounds above a certain level. Another example: The user can select to be alerted only if the system recognizes that the sounds are from a police or ambulance siren.Furthermore, the user can create different environment profiles that correspond to different ambient noise levels. For example, the user can have a work profile, a traffic profile, and a parking profile, with the volume threshold for the traffic profile being higher than the volume threshold for the parking profile.

[0029] Procedure 500 proceeds to step 530, in which it receives a signal indicating a sound detected by the sound sensors. In step 540, Procedure 500 determines whether the detected sound data indicates the presence of a potentially hazardous event. This determination can be based on sound data exceeding a threshold (e.g., exceeding a certain level) or on structure detection. For example, Procedure 500 can use structure detection to determine that the sound is the siren of a police car, an ambulance, or a fire engine. Other examples include using structure detection to identify sounds from heavy machinery (e.g., a reversing alarm), sounds from barking dogs, or verbal warnings (e.g., "STOP," "WAIT," "FIRE").In some embodiments, the determination can be made based on a combination of structure detection and a threshold. For example, method 500 can determine that the detected sound data indicates the presence of a potentially hazardous event if the detected sound is a verbal warning with a volume greater than a threshold. The structure detection can be performed using artificial intelligence. The threshold can be adjusted by the user in step 520.

[0030] If the system determines that the sound data does not indicate the presence of a potentially hazardous event, procedure 500 returns to step 530 to continue sound data detection without activating the alarm element. If procedure 500 determines that the sound data indicates the presence of a potentially hazardous event, it proceeds to step 550. In step 550, procedure 500 analyzes the sound data and determines its direction and / or intensity. Procedure 500 may make this determination using a multi-channel compensation method. Procedure 500 also determines that the sound data detected by multiple sound sensors corresponds to the same sound, allowing it to determine the direction of the sound data.For example, method 500 can search for an substantially similar waveform detected by one or more sound sensors and then perform a multi-channel compensation procedure based on the sound data.

[0031] In step 560, procedure 500 activates the alarm elements to provide the user with tactile feedback regarding the presence, direction, and / or intensity of the detected sound data. The use of this system increases the user's environmental awareness. Procedure 500 then returns to step 530 to continue acquiring sound data.

[0032] Although in Fig. 5. A specific number of operations related to procedure 500 are shown; procedure 500 can be performed with more or fewer operations than shown. Fig. 5 will be executed. Even if in Fig. 5. If a specific sequence of operations is specified with respect to procedure 500, the operations that procedure 500 has may be performed in any suitable order.

[0033] Fig. Figure 6 is an example graphic for the operation of an example system for providing spatial audio detection and tactile feedback, according to the examples in the present disclosure. Fig. Section 5 provides further details of Procedure 500.

[0034] As described in step 520, the user can configure the threshold based on a decibel level, structure detection, or a combination thereof. In step 540, if the sound detected in step 530 is below the threshold, procedure 500 operates in normal mode, with no alarm elements active. However, if the system detects in step 540 that the sound is above the threshold, procedure 500 operates in Activation to Sound mode, activating one or more alarm elements to alert the user to the presence, direction, and / or intensity of a potential hazard.

[0035] Although the examples described above, other variations and examples can be derived from this revelation without deviating from the spirit and scope of protection of these examples. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 531,861

[0001]

Claims

[1] System that features: a portable device; a large number of sound sensors installed in the portable device; an alarm element installed in the portable device; and a controller that is coupled with the multitude of sound sensors and the alarm element, the controller serving to: to receive a signal indicating that a sound has been detected by at least one of the multitude of sound sensors; to determine whether the sound indicates the presence of a potentially dangerous event; to determine the direction of the sound based on the received signal, which provides an indication of the sound; If the sound indicates the presence of the potentially dangerous event, activate the alarm element; and If the sound does not indicate the presence of the potentially dangerous event, do not activate the alarm element. [2] System according to claim 1, wherein the alarm element comprises a plurality of alarm elements, and the activation of the alarm element comprises the activation of one or more of the plurality of alarm elements to indicate a direction of the sound. [3] System according to one of claims 1 to 2, wherein the activation of the alarm element comprises indicating an intensity of the sound. [4] System according to any one of claims 1 to 3, wherein the portable device is worn on the head or upper body of a user to enable spatial directional notification. [5] System according to any one of claims 1 to 4, wherein the plurality of sound sensors includes at least three sound sensors, each of which has a primary detection contribution for a sector around the portable device. [6] System according to any one of claims 1 to 5, wherein determining whether the sound sensor indicates the presence of a potentially dangerous event is based on the detection of a keyword. [7] System according to one of claims 1 to 6, wherein the determination of a direction of the sound based on data from the plurality of sound sensors comprises a multi-channel compensation method. [8] Method which features: Receiving a signal indicating a sound detected by at least one of a plurality of sound sensors; Determine whether the sound indicates the presence of a potentially dangerous event; Determining the direction of sound based on the received signal indicating the sound; if the sound indicates the presence of the potentially dangerous event, to activate an alarm element in a portable device; and If the sound does not indicate the presence of the potentially dangerous event, do not activate the alarm element in the portable device. [9] Method according to claim 8, wherein the activation of the alarm element comprises the activation of a plurality of alarm elements to indicate a direction of the sound. [10] Method according to one of claims 8 to 9, wherein the activation of the alarm element comprises specifying an intensity of the sound. [11] Method according to any one of claims 8 to 10, wherein the portable device is worn on the head or upper body of a user to enable spatial directional notification. [12] Method according to any one of claims 8 to 11, wherein the plurality of sound sensors includes at least three sound sensors, wherein each of the at least three sound sensors has a primary detection contribution for a sector around the portable device. [13] Method according to any one of claims 8 to 12, wherein determining whether the sound sensor indicates the presence of a potentially dangerous event is based on the detection of a keyword. [14] Method according to any one of claims 8 to 13, wherein determining the direction of the sound based on the received signal indicating the sound comprises the use of a multi-channel compensation method. [15] Device comprising: a sound sensor interface; an alarm element interface; and a controller circuit to: to receive a signal indicating a sound detected by at least one of a plurality of sound sensors in a portable device; to determine whether the sound indicates the presence of a potentially dangerous event; to determine the direction of the sound based on the received signal indicating the sound; and if the sound indicates the presence of the potentially dangerous event, activate an alarm element in the portable device; and If the sound does not indicate the presence of the potentially dangerous event, do not activate the alarm element in the portable device. [16] Device according to claim 15, wherein the activation of the alarm element comprises the activation of a plurality of alarm elements to indicate a direction of the sound. [17] Device according to one of claims 15 to 16, wherein the activation of the alarm element comprises indicating an intensity of the sound. [18] Device according to any one of claims 15 to 17, wherein the plurality of sound sensors includes at least three sound sensors, wherein the respective sound sensors have a primary detection contribution for a sector around the portable device. [19] Device according to any one of claims 15 to 18, wherein the determination of whether the sound sensor indicates the presence of a potentially dangerous event is based on the recognition of a keyword. [20] Device according to one of claims 15 to 19, wherein the determination of a direction of the sound based on data from the plurality of sound sensors comprises a multi-channel compensation method.

Images