Sensory control device and method

Abstract

A sensory control device according to an embodiment includes: an operation acquisition unit that acquires operation content to the auricle of a user, the operation content being related to the user's intention to change a stimulus to the hearing of the user; and a control unit that controls driving of a movable member that changes a stimulus to the hearing of the user on the basis of the operation content acquired by the operation acquisition unit.

Description

Sensory control device and method

[0001] FIELD OF THE INVENTION Embodiments of the present invention relate to sensory control devices and methods.

[0002] Techniques have been proposed for controlling perceptual abilities and expanding, for example enhancing, attenuating, or transforming them, by modifying stimuli input to the body. For example, as disclosed in Patent Literature 1, a technique has been proposed for modifying auditory perception by replacing stimuli input to the user's ears with manipulated stimuli.

[0003] Japanese Patent Application Publication No. 2015-108766

[0004] However, the technique disclosed in Patent Document 1 requires the user to wear the device in a manner that blocks the ear canal.

[0005] This has raised concerns that the device causes inconvenience to users by constantly wearing it, blocking the ear canal, and poses a risk to users of developing otitis externa and other conditions.

[0006] This invention was made in light of the above circumstances, and its purpose is to provide a sensory control device and method that can appropriately transform the stimulation to the user's hearing.

[0007] A sensory control device according to one embodiment of the present invention includes an action acquisition unit that acquires the content of an action performed on a user's auricle, which is related to the user's intention to change the stimulus to the user's hearing, and a control unit that controls the drive of a movable member that changes the stimulus to the user's hearing, based on the content of the action acquired by the action acquisition unit.

[0008] A sensory control method according to one aspect of the present invention is a method performed by a sensory control device, and includes: an operation acquisition unit of the sensory control device acquiring the content of an operation performed on the user's auricle, which is related to the user's intention to change the stimulus to the user's hearing; and a control unit of the sensory control device controlling the drive of a movable member that changes the stimulus to the user's hearing, based on the content of the operation acquired by the operation acquisition unit.

[0009] According to the present invention, it is possible to appropriately change the stimulation to the user's hearing.

[0010] FIG. 1 is a diagram showing an application example of a sensation control device according to a first embodiment of the present invention. FIG. 2 is a diagram showing an example of information stored in a control DB according to the first embodiment of the present invention. FIG. 3 is a flowchart showing an example of processing operations by the sensation control device according to the first embodiment of the present invention. FIG. 4 is a diagram showing an application example of a sensation control device according to a second embodiment of the present invention. FIG. 5 is a diagram showing an example of information stored in a control DB according to the second embodiment of the present invention. FIG. 6 is a flowchart showing an example of processing operations by the sensation control device according to the second embodiment of the present invention. FIG. 7 is a block diagram showing an example of the hardware configuration of a sensation control device according to an embodiment of the present invention.

[0011] An embodiment of the present invention will be described below with reference to the drawings. In this embodiment, the user's perception is controlled by changing the transmission path of the stimulus from the sound source to the user's hearing, rather than replacing the stimulus itself with another stimulus. For example, the user's body can be transformed to change the stimulus input to the user's eardrum without blocking the ear canal.

[0012] In addition, in this embodiment, for example, the shape of the user's auricle can be artificially widened using a movable member (hereinafter sometimes simply referred to as a member) attached to the user's head, for example, at the back of the auricle, thereby increasing the amount of stimulation input to the user's eardrum, thereby reducing sound leakage to the user's surroundings.

[0013] In addition, in this embodiment, the artificially enlarged pinna can be made of an acoustic metamaterial, i.e., a material that attenuates or reflects only specific frequencies, thereby further expanding the range of design options. Furthermore, in this embodiment, the amount of stimulation input to the user's eardrum can be reduced by covering the pinna with a movable member. In this embodiment, the proportion of the pinna covered by the member can be changed between the left and right ears, which can affect the directionality of sound collection or sound source localization.

[0014] In this embodiment, the amount of stimulation can be increased or decreased by providing feedback in response to a user's customary and intuitive gestures, such as "placing a hand over an ear when it is difficult to hear" or "covering an ear with a hand when it is not desired to hear."

[0015] In this way, in this embodiment, the user's auditory perception can be transformed without blocking the user's ear canal, thereby avoiding the hassle of blocking the ear canal with a device and eliminating the risk of otitis externa.

[0016] (First embodiment) Next, a first embodiment of the present invention will be described. Fig. 1 is a diagram showing an application example of a sensation control device according to a first embodiment of the present invention. As shown in Fig. 1, a sensation control device 100 according to the first embodiment of the present invention includes a control DB (database) 11, a gesture acquisition unit 12, a control determination unit 13, and a drive unit 14.

[0017] 2 is a diagram showing an example of information stored in a control DB according to the first embodiment of the present invention. As shown in Fig. 2, the control DB 11 according to the first embodiment of the present invention stores control information that defines control content according to the content of a user's gesture.

[0018] The gesture information in this control DB 11 is the content of the user's actions on the pinna, related to the user's intention to change the stimulation to the auditory sense, and can be determined, for example, by a time-series vector-based learning model.

[0019] In one example shown in Figure 2, in response to a user gesture of "placing a hand to an ear," which is related to the user's intention to increase the stimulation to their hearing, it is defined that the appropriate control is to "spread the pinna," i.e., to artificially expand the area of ​​the user's pinna using a movable member.

[0020] In another example shown in Figure 2, in response to a user gesture of "covering the ear with the hand placed over the ear," which is related to the user's intention to attenuate auditory stimulation, the appropriate control is defined as "spreading the pinna and covering the ear," that is, moving or deforming a movable member attached to the user to cover the pinna without blocking the user's ear canal.

[0021] The gesture acquisition unit 12 acquires gestures made by the user using, for example, a touch sensor or a vision-based method using a camera. The control determination unit 13 determines the control content of the user's outer ear device based on the content of the gesture acquired by the gesture acquisition unit 12 and the control information stored in the control DB 11. This device is, for example, a device used in the field of soft robotics, and changes the stimulation to the user's ear canal by expanding the shape and area of ​​a movable member made of a polymer material such as silicone attached to the back of the user's auricle using air pressure or the like in response to a drive signal, or by deforming this member using air pressure or the like so that the area near the user's auricle is covered.

[0022] Furthermore, the above-mentioned device that changes the stimulation to the user's ear canal is not limited to the above, and may be, for example, a device in which a movable member is attached directly to the user's auricle, and when attenuating the stimulation to the user's ear canal, the user's auricle is positioned near the ear canal so that it blocks the ear canal.

[0023] The driver 14 drives the member by outputting a drive signal to the ear canal device according to the control content determined by the control determination unit 13. The operation of the driver 14 can drive a servo motor in the ear canal device to expand the member or deform the member so that the area around the user's ear canal is covered.

[0024] 3 is a flowchart showing an example of processing operations performed by the sensory control device according to the first embodiment of the present invention. First, the gesture acquisition unit 12 acquires the contents of a user's gesture input from an external device (S11). The control determination unit 13 determines the control contents of the earbud device based on the gesture acquired by the gesture acquisition unit 12 and the control information stored in the control DB 11, and transmits a drive signal for the earbud device according to the determined result to the drive unit 14 (S12). The drive unit 14 receives the drive signal transmitted in S12 and drives the earbud device (S13).

[0025] Second Embodiment Next, a second embodiment of the present invention will be described. Fig. 4 is a diagram showing an application example of a sensation control device according to the second embodiment of the present invention. As shown in Fig. 4, the sensation control device 100a according to the second embodiment of the present invention includes a touch acquisition unit 21, a press-down pressure acquisition unit 22, and an audio stimulus amount acquisition unit 23 in addition to the control DB 11, gesture acquisition unit 12, control determination unit 13, and drive unit 14 described in the first embodiment.

[0026] Fig. 5 is a diagram showing an example of information stored in a control DB according to the second embodiment of the present invention. As shown in Fig. 5, the control DB 11 according to the second embodiment of the present invention stores control information that defines control content and a target audio stimulus amount (sometimes referred to as a target audio stimulus value) according to the content of a user's gesture.

[0027] In one example shown in Figure 5, in response to a user gesture of "placing hands to ears," a control of "spreading auricles" is performed, and it is defined that this control is appropriate to increase the amount of audio stimulation (sometimes referred to as audio stimulation value) to the right and left ears by 10 dB.

[0028] In another example shown in Figure 5, in response to a user gesture of "applying pressure to the right ear with the hand placed on the right ear," the appropriate control is defined as "expanding the right auricle to cover the right ear," i.e., artificially expanding the area of ​​the user's right auricle using a component of the ear device, and then moving or deforming this component to cover the area near the auricle of the right ear without blocking the ear canal of the user's right ear, and attenuating the amount of audio stimulation to the right ear by 10 dB, while leaving the amount of audio stimulation to the left ear unchanged.

[0029] The touch acquisition unit 21 acquires, for example, a capacitive touch action by the user placing a hand near the auricle. The pressing pressure acquisition unit 22 acquires the strength of the pressure of the user's touch, i.e., the strength of the pressing pressure, using a pressure sensor.

[0030] The audio stimulus amount acquisition unit 23 acquires the current audio stimulus amount to the user's ear canals using microphones or the like attached near the ear canals of the right and left ears.

[0031] The control determination unit 13 according to the second embodiment first determines the control content of the user's external ear device and the target audio stimulus amount associated with this control content based on the gesture content acquired by the gesture acquisition unit 12, the control information stored in the control DB 11, and the audio stimulus amount acquired by the audio stimulus amount acquisition unit 23. In this embodiment, the target audio stimulus amount is a target value for increasing or decreasing the audio stimulus amount after the movable member is driven in accordance with the determined control content, relative to the audio stimulus amount before the movable member is driven in accordance with the control content. This target value corresponds to the difference between the audio stimulus amount before driving and the audio stimulus amount after driving.

[0032] For example, when the content of the gesture acquired by the gesture acquisition unit 12 indicates "putting hands to ears," which is one example shown in Figure 5, "spreading the auricles" associated with this "putting hands to ears" is determined as the control content, and "right: increase by 10 dB" and "left: increase by 10 dB" associated with the above "putting hands to ears" are determined as the target audio stimulation amounts related to the control content.

[0033] Furthermore, when the content of the gesture acquired by the gesture acquisition unit 12 indicates another example shown in FIG. 5, "applying pressure to the right ear with the hand held to the right ear," "spreading the right auricle to cover the right ear," which is associated with this "applying pressure to the right ear with the hand held to the right ear," is determined as the control content, and "right: 10 dB increase" and "left: none," which are associated with the above "applying pressure to the right ear with the hand held to the right ear," are determined as the target audio stimulation amounts related to the control content.

[0034] In addition, the control decision unit 13 in the second embodiment compares the current audio stimulus amount acquired by the audio stimulus amount acquisition unit 23 with the target audio stimulus amount related to the above-determined control content among the target audio stimulus amounts stored in the control DB 11, and performs feedback control according to the result of this comparison.

[0035] 6 is a flowchart showing an example of processing operations by the sensory control device according to the second embodiment of the present invention. First, the touch acquisition unit 21 acquires a touch action by the user near the auricle, and outputs information indicating the acquisition to the gesture acquisition unit 12. Furthermore, the pressing pressure acquisition unit 22 acquires the strength of the pressure of the user's touch, and outputs pressing pressure information indicating this strength to the gesture acquisition unit 12.

[0036] The gesture acquisition unit 12 acquires touch information acquired from the touch acquisition unit 21 and pressure information acquired from the pressure acquisition unit 22 as the content of the user's gesture (S21). The audio stimulation amount acquisition unit 23 acquires the audio stimulation amounts for the user's right ear and left ear, respectively (S22).

[0037] The control determination unit 13 determines the control content and the target sound stimulation amount of the external ear device based on the gesture acquired by the gesture acquisition unit 12 and the content of the control information stored in the control DB 11, and transmits a drive signal for the external ear device according to the determined result to the drive unit 14 (S23). The drive unit 14 receives the drive signal transmitted in S23 and drives the external ear device (S24).

[0038] After this driving, the audio stimulus amount acquisition unit 23 acquires the current audio stimulus amounts for the right ear and the left ear (S25).

[0039] The control decision unit 13 compares the amount of audio stimulation obtained in S22, i.e., the amount of audio stimulation before driving by the drive unit 14, with the amount of audio stimulation obtained in S25, i.e., the amount of audio stimulation after driving by the drive unit 14, for the left and right ears, and determines whether the stimulation based on the target amount of audio stimulation determined in S23 has been achieved (S26).

[0040] If the result of S26 is "No," the control decision unit 13 determines the control amount for the external ear device so that the target amount of audio stimulation can be achieved, transmits a drive signal for the external ear device according to this determination result to the drive unit 14 (S27), and returns to S24.

[0041] For example, in S23, when "expand pinna" is determined as the control content, and "right: 10 dB increase" and "left: 10 dB increase" associated with the "hand to ear" are determined as the target audio stimulus amounts related to the control content, if the audio stimulus amount for the right ear after the drive does not satisfy the condition of "right: 10 dB increase" relative to the audio stimulus amount for the right ear before the drive, or if the audio stimulus amount for the left ear after the drive does not satisfy the condition of "left: 10 dB increase" relative to the audio stimulus amount for the left ear before the drive, a control amount is determined in S27 such that the area of ​​the pinna of the ear related to the unsatisfied condition is artificially further expanded to increase the audio stimulus amount after the drive. If the determination in S26 is "Yes," the series of processes ends.

[0042] Fig. 7 is a block diagram showing an example of the hardware configuration of a sensation control device according to one embodiment of the present invention. In the example shown in Fig. 7, the sensation control device 100 according to the first embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 111A such as a CPU (Central Processing Unit). A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to this hardware processor 111A via a bus 115. The same applies to the sensation control device 100a shown in Fig. 4.

[0043] The communication interface 114 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network NW. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0044] An input device 200 and an output device 300, which are attached to the sensation control device 100 and used by a user or the like, are connected to the input / output interface 113. The input / output interface 113 takes in operation data input by a user or the like through the input device 200, such as a keyboard, touch panel, touchpad, or mouse, and outputs the output data to an output device 300, which includes a display device using liquid crystal or organic electroluminescence (EL) or the like, for display. The input device 200 and the output device 300 may be devices built into the sensation control device 100, or may be input devices and output devices of other information terminals that can communicate with the sensation control device 100 via a network NW.

[0045] The program memory 111B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and stores programs necessary to execute various control processes, etc., according to one embodiment.

[0046] The data memory 112 is a tangible storage medium that is, for example, a combination of the above-mentioned non-volatile memory and a volatile memory such as RAM (Random Access Memory), and is used to store various data acquired and created during various processes performed by the sensory control device 100.

[0047] The sensation control device 100 according to one embodiment of the present invention may be configured as a data processing system or information processing device having software-based processing function units. The storage areas used as work memory by each unit of the sensation control device 100 may be configured using the data memory 112 shown in FIG. 7. However, these configured storage areas are not essential components within the sensation control device 100, and may be areas provided in a storage device such as an external storage medium such as a USB (Universal Serial Bus) memory, or a database server located in the cloud.

[0048] The processing function unit can be realized by having the hardware processor 111A read and execute a program stored in the program memory 111B, but the processing function unit may also be realized in various other forms, including an integrated circuit such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0049] The methods described in each embodiment may be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), or may be transmitted and distributed via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only executable programs but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-described processing by having the operation controlled by this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0050] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0051] DESCRIPTION OF SYMBOLS 100, 100a... Sense control device 11... Control DB 12... Gesture acquisition unit 13... Control determination unit 14... Driving unit 21... Touch acquisition unit 22... Pressing pressure acquisition unit 23... Audio stimulation amount acquisition unit

Claims

1. A sensory control device comprising: an action acquisition unit that acquires action content of a user on the auricle related to the user's intention to change the stimulus to the user's hearing; and a control unit that controls the drive of a movable member that transforms the stimulus to the user's hearing based on the action content acquired by the action acquisition unit.

2. The sensory control device according to claim 1, wherein the action acquisition unit acquires the content of the action performed on the auricle based on the user's touch action on the auricle and the strength of the pressure applied by the user when touching the auricle.

3. A sensory control device as described in claim 1, further comprising an audio stimulus acquisition unit that acquires an amount of audio stimulus to the user, wherein the control unit controls the drive of the movable member so that the amount of audio stimulus acquired by the audio stimulus acquisition unit before controlling the drive of the movable member and the amount of audio stimulus acquired by the audio stimulus acquisition unit after controlling the drive of the movable member satisfy a condition based on the operation content acquired by the operation acquisition unit.

4. A method performed by a sensory control device, comprising: acquiring, by an operation acquisition unit of the sensory control device, operation content of the user's auricle related to the user's intention to change the stimulus to the user's hearing; and controlling, by a control unit of the sensory control device, the drive of a movable member that transforms the stimulus to the user's hearing based on the operation content acquired by the operation acquisition unit.

Images