Signal processing device, congnitive function improvement system, signal processing method, and program

Abstract

A signal processing device according to an aspect of the present disclosure comprises: an output means that outputs an acoustic signal which is generated by amplitude modulation and has an amplitude change corresponding to the frequency of a gamma wave; a storage means in which information of the acoustic signal output by the output means is stored; and a transmission means that transmits a notification based on the information stored in the storage means to an external device.

Description

Signal processing device, cognitive function improvement system, signal processing method, and program

[0001] The present disclosure relates to a signal processing device, a cognitive function improvement system, a signal processing method, and a program.

[0002] There is a research report that suggests that inducing gamma waves in the brain of an organism by subjecting it to pulsed sound stimuli at a frequency of about 40 times per second is effective in improving the organism's cognitive function (see Non-Patent Document 1). Gamma waves refer to neural oscillations that capture periodic neural activity in the brain's cortex using electrophysiological techniques such as electroencephalography and magnetoencephalography, and that have frequencies in the gamma band (25-140 Hz).

[0003] Patent document 1 discloses adjusting the volume by increasing or decreasing the amplitude of a sound wave or sound track to create a rhythmic stimulus corresponding to a stimulus frequency that induces brain wave entrainment.

[0004] Special Publication No. 2020-501853

[0005] Multi-sensory Gamma Stimulation Ameliorates Alzheimer's-Associated Pathology and Improves Cognition Cell 2019 Apr 4;177(2):256-271.e22. doi: 10.1016 / j.cell.2019.02.014.

[0006] When a user listening to sounds using amplitude-modulated acoustic signals in the hope of improving cognitive function is listened to, relevant parties may want to understand the user's listening status. For example, the user's family may feel uneasy if they cannot confirm that the user is listening to the sounds. Medical professionals who perform treatment using listening to sounds cannot provide appropriate treatment instructions unless they can confirm whether the user (patient) is listening to the sounds. If the user does not have an opportunity to review their listening status if they forget to listen to the sounds, it may be difficult for them to make listening to the sounds a habit.

[0007] An object of the present disclosure is to provide a technology that enables related parties to understand the listening conditions of a user with respect to a sound corresponding to an amplitude-modulated acoustic signal.

[0008] A signal processing device according to one aspect of the present disclosure includes an output means for outputting an acoustic signal generated by amplitude modulation, the acoustic signal having an amplitude change corresponding to the frequency of gamma waves, a memory means for storing information about the acoustic signal output by the output means, and a transmission means for transmitting a notification based on the information stored by the memory means to a device external to the signal processing device.

[0009] 1 is a block diagram showing the configuration of an acoustic system of the present embodiment. FIG. 2 is a block diagram showing the configuration of a signal processing device of the present embodiment. FIG. 3 is a block diagram showing the configuration of a related party terminal of the present embodiment. FIG. 4 is an explanatory diagram of one aspect of the present embodiment. FIG. 5 is a diagram showing the data structure of a listening history database of the present embodiment. FIG. 6 is a flowchart of acoustic signal processing of the present embodiment. FIG. 7 is a flowchart of information notification processing of the present embodiment. FIG. 8 is a diagram showing an example of a score function used in the information notification processing of the present embodiment. FIG. 9 is a diagram showing an example of presentation of target information in the information notification processing of the present embodiment. FIG. 10 is a diagram showing an example of presentation of target information in the information notification processing of the present embodiment. FIG. 11 is a flowchart of information notification processing of modified example 1. FIG. 12 is a block diagram showing the configuration of an acoustic system of modified example 2. FIG. 13 is a flowchart of acoustic signal processing of modified example 2.

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the drawings for explaining the embodiment, the same components are generally designated by the same reference numerals, and repeated description thereof will be omitted.

[0011] (1) Configuration of the Acoustic System The configuration of the acoustic system will be described below. Fig. 1 is a block diagram showing the configuration of the acoustic system of this embodiment.

[0012] As shown in FIG. 1, the sound system 1 includes a signal processing device 10, a sound output device 30, a sound source device 50, and a related party terminal 70.

[0013] The signal processing device 10 and the sound source device 50 are connected to each other via a predetermined interface capable of transmitting an audio signal. The interface is, for example, an SPDIF (Sony Philips Digital Interface), an HDMI (registered trademark) (High-Definition Multimedia Interface), a pin connector (RCA pin), or an audio interface for headphones. The interface may also be a wireless interface using Bluetooth (registered trademark), etc. The signal processing device 10 and the audio output device 30 are similarly connected to each other via a predetermined interface. The audio signal in this embodiment includes either or both of an analog signal and a digital signal.

[0014] The signal processing device 10 and the related party terminal 70 may be connected to each other via a network (e.g., the Internet or an intranet) or via a predetermined interface (e.g., a wireless interface using Bluetooth, etc.). Note that the signal processing device 10 and the related party terminal 70 may also be connected to each other via an external device (e.g., a server) not shown.

[0015] The signal processing device 10 performs acoustic signal processing on an input acoustic signal acquired from a sound source device 50. The acoustic signal processing by the signal processing device 10 includes at least modulation processing of the acoustic signal (details of which will be described later). The acoustic signal processing by the signal processing device 10 may also include conversion processing of the acoustic signal (e.g., separation, extraction, or synthesis). Furthermore, the acoustic signal processing by the signal processing device 10 may further include amplification processing of the acoustic signal similar to that of an AV amplifier, for example. The signal processing device 10 sends an output acoustic signal generated by the acoustic signal processing to the acoustic output device 30. The signal processing device 10 is an example of an information processing device.

[0016] The sound output device 30 generates sound according to the output sound signal acquired from the signal processing device 10. The sound output device 30 is, for example, a loudspeaker (which may include a speaker with a built-in amplifier (powered speaker)), headphones, earphones, or a smart speaker. The sound output device 30 can also be configured as a single device together with the signal processing device 10. Specifically, the signal processing device 10 and the sound output device 30 can be implemented in a TV, radio, music player, AV amplifier, speaker, headphones, earphones, smartphone, PC (including tablet PCs; the same applies below), or smart speaker. The signal processing device 10 and the sound output device 30 constitute a cognitive function improvement system.

[0017] The sound source device 50 sends an input acoustic signal to the signal processing device 10. The sound source device 50 is, for example, a TV, a radio, a music player, a smartphone, a PC, an electronic musical instrument, a telephone, a game console, an amusement machine, or a device that transmits an acoustic signal via broadcasting or information communication.

[0018] The related party terminal 70 can be implemented as a smartphone or a PC. The related party terminal 70 has a function of receiving information from the signal processing device 10 and presenting the notified information to the user (related party) of the related party terminal 70.

[0019] In the following description, a related party refers to at least one of the following people, for example: a user of the sound output device 30 (i.e., a person who listens to sounds to improve cognitive function, hereinafter referred to as the "patient"); a family member, friend, or acquaintance of the patient; a medical professional (e.g., the patient's doctor, nurse, or medical technician); a caregiver (e.g., the patient's caregiver); a creator or provider of content corresponding to the input audio signal (described below); a creator or provider of content corresponding to the input audio signal; a provider of the signal processing device 10; an administrator of a facility used by the patient; an insurance company; other patients in the same community as the patient; a system provider; and a third party designated by the above-mentioned related party. The related party terminal 70 is owned by a related party. "Owned" here does not necessarily mean ownership, but may also mean temporary exclusive use for operation. The related party terminal 70 may be shared by multiple related parties. Furthermore, the signal processing device 10 may notify multiple related party terminals 70 of information.

[0020] (1-1) Configuration of the Signal Processing Device The configuration of the signal processing device will be described below. Fig. 2 is a block diagram showing the configuration of the signal processing device of this embodiment.

[0021] 2, the signal processing device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14. The signal processing device 10 is connected to a display 21.

[0022] The storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of a read-only memory (ROM), a random access memory (RAM), and storage (for example, a flash memory or a hard disk). The programs and data may be provided via a network, or may be provided by being recorded on a computer-readable recording medium.

[0023] The programs include, for example, the following: OS (Operating System) programs; and application programs that execute information processing.

[0024] The data includes, for example, the following data: Databases referenced in information processing Data obtained by executing information processing (i.e., execution results of information processing)

[0025] The processor 12 is a computer that realizes the functions of the signal processing device 10 by reading and executing programs stored in the storage device 11. Note that at least a part of the functions of the signal processing device 10 may be realized by one or more dedicated circuits. The processor 12 is, for example, at least one of the following: CPU (Central Processing Unit) GPU (Graphic Processing Unit) ASIC (Application Specific Integrated Circuit) FPGA (Field Programmable Array) DSP (Digital Signal Processor)

[0026] The input / output interface 13 is configured to acquire information (e.g., user instructions) from an input device connected to the signal processing device 10, and output information (e.g., an image or a control signal) to an output device connected to the signal processing device 10. The input device is, for example, a sound source device 50, a physical button, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display 21, an audio output device 30, a light-emitting element, a speaker different from the audio output device 30, or a combination thereof.

[0027] Furthermore, the input / output interface 13 may include signal processing hardware such as an A / D converter, a D / A converter, an amplifier, a mixer, a filter, and the like.

[0028] The communication interface 14 is configured to control communication between the signal processing device 10 and an external device (for example, the sound output device 30, the sound source device 50, or the participant terminal 70).

[0029] The display 21 is configured to display an image (still image or moving image). The display 21 is, for example, a liquid crystal display or an organic EL display. Alternatively, the display 21 may be a seven-segment display.

[0030] (1-2) Configuration of the Related Party Terminal The configuration of the related party terminal will be described below. Fig. 3 is a block diagram showing the configuration of the related party terminal of this embodiment.

[0031] 3, the related party terminal 70 includes a storage device 71, a processor 72, an input / output interface 73, and a communication interface 74. The related party terminal 70 is connected to a display 81.

[0032] The storage device 71 is configured to store programs and data. The storage device 71 is, for example, a combination of ROM, RAM, and storage (for example, flash memory or a hard disk). The programs and data may be provided via a network, or may be provided by being recorded on a computer-readable recording medium.

[0033] The programs include, for example, the following programs: OS programs; application programs for executing information processing.

[0034] The data includes, for example, the following data: Databases referenced in information processing Execution results of information processing

[0035] The processor 72 is a computer that realizes the functions of the involved party terminal 70 by reading and executing the programs stored in the storage device 71. At least a part of the functions of the involved party terminal 70 may be realized by one or more dedicated circuits. The processor 72 is, for example, at least one of the following: CPU GPU ASIC FPGA DSP

[0036] The input / output interface 73 is configured to acquire information (e.g., user instructions) from an input device connected to the related party terminal 70, and output information (e.g., an image) to an output device connected to the related party terminal 70. The input device is, for example, a physical button, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display 81, a speaker, or a combination thereof.

[0037] The communication interface 74 is configured to control communication between the participant terminal 70 and an external device (for example, the signal processing device 10).

[0038] The display 81 is configured to display an image (still image or moving image) and is, for example, a liquid crystal display or an organic EL display.

[0039] (2) One aspect of the embodiment One aspect of the present embodiment will be described below. Fig. 4 is an explanatory diagram of one aspect of the present embodiment.

[0040] As shown in FIG. 4 , the signal processing device 10 acquires an acoustic signal (input acoustic signal) from the sound source device 50. The signal processing device 10 performs amplitude modulation processing on the acoustic signal. The signal processing device 10 outputs the generated acoustic signal (output acoustic signal) to the acoustic output device 30. Because the output acoustic signal has amplitude changes corresponding to the frequency of gamma waves, when the patient PA1 listens to the sound generated by the output acoustic signal, gamma waves are induced in the brain of the patient PA1 (brain waves are synchronized with the gamma frequency). This is expected to improve cognitive function (e.g., treat or prevent dementia). Meanwhile, the signal processing device 10 stores the patient PA1's listening history (an example of "information on the output acoustic signal") for the sound for improving cognitive function (the sound generated by the acoustic output device 30 in response to the output acoustic signal). As an example, the signal processing device 10 stores information in a listening history database (described later) regarding what kind of amplitude modulation was performed on the acoustic signal having what characteristic parameters to generate the output acoustic signal, as well as information regarding the playback volume and duration at which the patient PA1 listened to the sound from the output acoustic signal.

[0041] The signal processing device 10 generates information (hereinafter referred to as "target information") regarding the hearing status of the patient PA1 for sounds for improving cognitive function based on the stored hearing history, and notifies the target information to the related party terminal 70. The related party terminal 70 presents the target information to the related party PC 2. This allows the related party PC 2 to understand the hearing status of the sound by the patient PA1.

[0042] This allows the related person PC 2 to obtain various effects according to the attributes of the related person PC 2. For example, a family member or a medical professional can encourage the patient PA1 to improve the hearing condition of the patient PA1 based on the notified target information.

[0043] Based on the notified target information, medical personnel can confirm whether patient PA1 is acting in accordance with the treatment plan, create or modify a treatment plan for patient PA1's cognitive function based on the hearing status, and make a diagnosis of patient PA1's cognitive function based on the hearing status. Based on the notified target information, patient PA1 can introspect on his or her own hearing status.

[0044] The facility manager can use the notified target information to efficiently manage the patient PA1 as a facility user, and the insurance company can use the notified target information to calculate a reasonable insurance premium for the patient PA1.

[0045] The provider of the signal processing device 10 can use the notified target information to conduct research and development to improve the cognitive function improvement effect of the signal processing device 10 (for example, a more effective method of generating and providing an output acoustic signal). The target information notified to the related party PC 2 may be anonymized. Furthermore, the patient may be allowed to freely choose whether or not to provide the target information in exchange for compensation (for example, a coupon).

[0046] The creator or provider of the content corresponding to the input audio signal can obtain suggestions for guidelines on creating or providing the content from the notified target information. The target information notified to the related party PC 2 may be anonymized. Furthermore, the patient may be free to choose whether or not to provide the target information in exchange for compensation (e.g., a coupon).

[0047] By viewing the notified target information, other patients belonging to the same community can check the behavioral records of their rival or fellow patients and reflect on their own behavioral records, making it easier to maintain motivation to continue listening to sounds to improve dementia.In addition, patients are aware that their behavioral records are being seen by other patients belonging to the same community, making it easier to maintain motivation to continue listening to sounds to improve dementia.

[0048] (3) Database The database of this embodiment will be described. The following databases are stored in the storage device 11. Alternatively, the databases may be stored in a storage device provided in an external device (e.g., a server) accessible by the signal processing device 10. In this case, information relating to a plurality of different signal processing devices 10 may be aggregated in one database.

[0049] (3-1) Listening History Database The listening history database of this embodiment will be described below. Fig. 5 is a diagram showing the data structure of the listening history database of this embodiment.

[0050] The listening history database stores listening history, which is log information about the patient's listening behavior to sounds for improving cognitive function.

[0051] 5, the listening history database includes an "ID" field, a "start date and time" field, an "end date and time" field, a "modulation method" field, a "feature value" field, and a "playback volume" field. Each field is associated with another field.

[0052] The "ID" field stores a listening history ID, which is information for identifying a listening history.

[0053] The "start date and time" field stores start date and time information, which is information about the date and time when the reproduction of the output acoustic signal started in the corresponding listening history.

[0054] The "End Date and Time" field stores end date and time information. The end date and time information is information about the date and time when playback of the output acoustic signal ended in the corresponding listening history. In other words, the period during which the patient listened to the sound of the output acoustic signal can be identified based on the start date and time information and the end date and time information.

[0055] The "Modulation Method" field stores modulation method information. The modulation method information is information about the processing performed to generate an output audio signal from an input audio signal in the corresponding listening history. For example, the modulation method information may include information about at least one of the following: - modulation function used for amplitude modulation - modulation depth - audio signal to which amplitude modulation has been applied - audio signal to which amplitude modulation has not been applied Here, when an intermediate audio signal (described in detail below) including multiple audio signals is generated based on an input audio signal and whether or not amplitude modulation has been applied to each of the multiple audio signals is individually determined, each of the multiple audio signals corresponds to either an "audio signal to which amplitude modulation has been applied" or an "audio signal to which amplitude modulation has not been applied." Similarly, when an input audio signal is made up of multiple audio signals and whether or not amplitude modulation has been applied to each of the multiple audio signals is individually determined, each of the multiple audio signals corresponds to either an "audio signal to which amplitude modulation has been applied" or an "audio signal to which amplitude modulation has not been applied." In these cases, since the function or modulation depth used for amplitude modulation may be different for each of the acoustic signals to which amplitude modulation is applied, the modulation method information may include information regarding the "function used for amplitude modulation" or "modulation depth" of each acoustic signal.

[0056] The "feature" field stores feature information. The feature information is information about the feature of the audio signal to which amplitude modulation has been applied in the corresponding listening history. For example, the feature information may include information about at least one of the following: - Distribution of amplitude, energy, or effective value in the frequency domain (e.g., the result of a fast Fourier transform (FFT)) - Distribution of amplitude, energy, or effective value in the time domain - Sound source type (e.g., vocal, instrument, object, music, conversation, natural sound, electronic sound, etc.) - Content attribute (e.g., program information of a TV program in which the audio signal is included)

[0057] The "playback volume" field stores playback volume information. The playback volume information is information about the playback volume of the output audio signal in the corresponding listening history. The signal processing device 10 may control the playback volume information by specifying the playback volume to the audio output device 30, or may acquire the playback volume information from the audio output device 30.

[0058] (4) Information Processing Information processing in this embodiment will be described.

[0059] (4-1) Acoustic Signal Processing The acoustic signal processing of this embodiment will be described below with reference to the flowchart of FIG.

[0060] The acoustic signal processing in FIG. 6 is realized by the processor 12 of the signal processing device 10 reading and executing a program stored in the storage device 11.

[0061] The acoustic signal processing in Fig. 6 starts when any of the following start conditions is met: The acoustic signal processing in Fig. 6 is called by another process or an external instruction. A related person performs an operation to call the acoustic signal processing in Fig. 6. The signal processing device 10 enters a predetermined state (e.g., power is turned on). A predetermined date and time arrives. A predetermined time has passed since a predetermined event (e.g., startup of the signal processing device 10 or the previous execution of the acoustic signal processing in Fig. 6).

[0062] 6, the signal processing device 10 acquires an input acoustic signal (S110). Specifically, the signal processing device 10 receives an input acoustic signal transmitted from the sound source device 50. In step S110, the signal processing device 10 may further perform A / D conversion of the input acoustic signal.

[0063] The input audio signal corresponds to, for example, at least one of the following: Musical content (e.g., singing, playing, or a combination thereof (i.e., a song). May include audio content accompanying video content.) Audio content (e.g., reading, narration, announcement, broadcast drama, solo performance, conversation, monologue, or a combination thereof. May include audio content accompanying video content.) Other audio content (e.g., electronic sounds, environmental sounds, or mechanical sounds). However, singing or audio content is not limited to sounds produced by human vocal organs, but may include sounds generated by voice synthesis technology.

[0064] After step S110, the signal processing device 10 generates an output acoustic signal (S111). Specifically, the signal processing device 10 generates the output acoustic signal by amplitude modulating at least a part of the input acoustic signal acquired in step S110.

[0065] As a first example of generating an output acoustic signal (S111), the signal processing device 10 generates an intermediate acoustic signal including a plurality of acoustic signals having different characteristics based on the input acoustic signal. Here, the characteristics may be determined based on an input operation by a person involved or an external instruction, or may be determined by an algorithm. For example, the signal processing device 10 may determine the characteristics for generating the intermediate acoustic signal based on the results of analyzing the input acoustic signal.

[0066] The features may be, for example, at least one of the following: sound features (particularly qualitative features); frequency features; time features; amplitude features; power features.

[0067] The signal processing device 10 selects one or more acoustic signals (hereinafter referred to as "target signals") to which amplitude modulation is to be applied from the multiple acoustic signals included in the intermediate acoustic signal. Which acoustic signal is selected as the target signal may be determined based on an input operation by a person involved, an external instruction, or an algorithm. For example, the signal processing device 10 may determine the target signal based on characteristics of the acoustic signal (balance between speech and music, volume change, type of music, timbre, or other characteristics). This allows the signal processing device 10 to select a target signal that will enhance the effect of modulation on improving cognitive function or that will minimize discomfort to the patient. However, the signal processing device 10 may also treat all of the multiple acoustic signals included in the intermediate acoustic signal as target signals.

[0068] The signal processing device 10 performs amplitude modulation on a selected target signal. As an example, the signal processing device 10 performs amplitude modulation on the target signal using a modulation function having a frequency corresponding to gamma waves (e.g., a frequency of 35 Hz to 45 Hz). Specifically, if a modulation function having a periodicity of 35 Hz to 45 Hz is A(t), a function representing the waveform of the acoustic signal before modulation is X(t), and a function representing the waveform of the modulated acoustic signal is Y(t), then Y(t) = A(t) · X(t). As a result, an amplitude change corresponding to the above frequency is added to the target signal. Here, the signal processing device 10 may determine a modulation function to be used for amplitude modulation of the target signal or a modulation depth for the amplitude modulation. The modulation function or modulation depth may be determined based on an input operation by a related person or an external instruction, or may be determined by an algorithm. Furthermore, the signal processing device 10 may determine a common modulation function or modulation depth for multiple target signals, or may determine a modulation function or modulation depth individually for multiple target signals.

[0069] The signal processing device 10 generates an output acoustic signal based on an acoustic signal (hereinafter referred to as a "non-target signal") that is not selected as a target signal from among the multiple acoustic signals included in the intermediate acoustic signal, and the modulated target signal. That is, the signal processing device 10 converts the non-target signal and the modulated target signal into an output acoustic signal. Specifically, the signal processing device 10 combines two or more acoustic signals from the non-target signal and the modulated target signal, or extracts or separates an acoustic signal from at least one of the non-target signal and the modulated target signal. The method of combining the acoustic signals is not limited, and may include, for example, signal summation, HRTF (Head Related Transfer Function) convolution, convolution of a transfer function that imparts position information of a sound source, or summation after performing these convolution processes. The signal processing device 10 may also perform at least one of amplification, volume adjustment, and D / A conversion of the output acoustic signal. On the other hand, if the asymmetric signal and the modulated target signal match the output format of the audio output device 30 (for example, if the asymmetric signal and the modulated target signal correspond to multi-channel audio signals associated with the speakers that make up a surround system serving as the audio output device 30), such conversion is unnecessary. In this case, the asymmetric signal and the modulated target signal are treated as output audio signals.

[0070] In a second example of generating an output acoustic signal (S111), the input acoustic signal includes a plurality of acoustic signals having different characteristics. The signal processing device 10 selects one or more acoustic signals to which amplitude modulation is to be applied from the plurality of acoustic signals included in the input acoustic signal. The second example of generating an output acoustic signal (S111) can be understood by appropriately replacing the "intermediate acoustic signal" with the "input acoustic signal" in the description of the first example above.

[0071] In a third example of generating an output acoustic signal (S111), the signal processing device 10 performs amplitude modulation on the input acoustic signal. The amplitude modulation on the input acoustic signal is similar to the amplitude modulation on the target signal described in the first example of generating an output acoustic signal (S111). As a result, an amplitude change corresponding to the frequency of gamma waves is added to the input acoustic signal. Here, the signal processing device 10 may determine a modulation function used for amplitude modulation of the input acoustic signal or a modulation depth of the amplitude modulation. The modulation function or modulation depth may be determined based on an input operation by a related person or an external instruction, or may be determined by an algorithm.

[0072] The signal processing device 10 generates an output acoustic signal based on a modulated input acoustic signal. That is, the signal processing device 10 converts the modulated input acoustic signal into an output acoustic signal. Specifically, when the modulated input acoustic signal consists of multiple acoustic signals, the signal processing device 10 combines two or more of the multiple acoustic signals, or extracts or separates an acoustic signal from the modulated input acoustic signal. Details of the acoustic signal combination method are as described in the first example of generating an output acoustic signal (S111). On the other hand, when the modulated input acoustic signal matches the output format of the acoustic output device 30 (for example, when the modulated input acoustic signal corresponds to a multi-channel acoustic signal associated with each speaker constituting a surround system as the acoustic output device 30), such conversion is not necessary. In this case, the modulated input acoustic signal is treated as the output acoustic signal.

[0073] After step S111, the signal processing device 10 transmits an output acoustic signal (S112). Specifically, the signal processing device 10 transmits the output acoustic signal generated in step S111 to the acoustic output device 30. The acoustic output device 30 generates a sound corresponding to the output acoustic signal.

[0074] After step S112, the signal processing device 10 executes saving of the listening history (S113). Specifically, the signal processing device 10 saves the listening history, including, for example, at least one of the following, in the listening history database (FIG. 5): Information on the feature amount of the acoustic signal to which amplitude modulation was applied in step S111 Information on the modulation method of the amplitude modulation performed in step S111 Information on the date and time when the acoustic output device 30 started playing the output acoustic signal sent in step S112 Information on the playback volume set in the acoustic output device 30 when playing the output acoustic signal sent in step S112 Information on the date and time when the acoustic output device 30 finished playing the output acoustic signal sent in step S112

[0075] The signal processing device 10 ends the audio signal processing of FIG. 6 at step S113. The signal processing device 10 may perform the audio signal processing of FIG. 6 all at once on an input audio signal having a certain playback period (e.g., one piece of music content), or may repeatedly perform the audio signal processing of FIG. 6 for each predetermined playback section of the input audio signal (e.g., every 100 ms). Alternatively, the signal processing device 10 may continuously perform modulation processing on the input audio signal, such as modulation using analog signal processing, and output a modulated audio signal. The audio signal processing shown in FIG. 6 may end in response to a specific end condition (e.g., a certain amount of time has elapsed, an operation has been performed by a relevant person, or the output history of modulated sound has reached a predetermined state).

[0076] (4-2) Information Notification Processing The information notification processing of this embodiment will be described. FIG. 7 is a flowchart of the information notification processing of this embodiment. FIG. 8 is a diagram showing an example of a score function used in the information notification processing of this embodiment. FIG. 9 is a diagram showing an example of presentation of target information in the information notification processing of this embodiment. FIG. 10 is a diagram showing an example of presentation of target information in the information notification processing of this embodiment. FIG. 11 is a diagram showing an example of presentation of target information in the information notification processing of this embodiment.

[0077] The information notification process of Fig. 7 starts when any of the following start conditions is met: The information notification process of Fig. 7 is called by another process or an external instruction. A related person performs an operation to call the information notification process of Fig. 7. The signal processing device 10 enters a predetermined state (e.g., power is turned on). A predetermined date and time arrives. A predetermined time has passed since a predetermined event (e.g., startup of the signal processing device 10, saving of a new listening history, or the previous execution of the information notification process of Fig. 7).

[0078] As shown in Fig. 7, the signal processing device 10 executes acquisition of a listening history (S120). Specifically, the signal processing device 10 acquires the patient's listening history from the listening history database (Fig. 5). As an example, the signal processing device 10 extracts from the listening history database a listening history in which at least one of the start date and time information or the end date and time information belongs to a period that is the target of target information generation (hereinafter referred to as the "target period"). Here, the target period may be any of the following: - The day, week, or month that includes the time when the information notification process of Fig. 7 was executed - A predetermined period going back from the time when the information notification process of Fig. 7 was executed - A period designated by a related person - A period designated according to an algorithm

[0079] After step S120, the signal processing device 10 generates target information (S121). Specifically, the signal processing device 10 generates target information based on the listening history acquired in step S120. The target information may include, for example, at least one of the following: Listening history (e.g., the period during which the patient listened to sounds for improving cognitive function) Information processed from the listening history (e.g., statistical information) A score related to the magnitude of the effect of improving cognitive function according to the sounds listened to by the patient.

[0080] For example, the signal processing device 10 can refer to a score function for deriving a score corresponding to elements included in the listening history (e.g., a listening period for a sound, a playback volume of an output audio signal, a modulation method for amplitude modulation applied to an audio signal, a feature amount of an audio signal to which amplitude modulation has been applied, or a combination thereof) in calculating the score. The score function may be stored in the storage device 11 or may be stored in a storage device included in an external device (e.g., a server) accessible by the signal processing device 10.

[0081] Here, the score represents the degree of improvement in cognitive function and is defined to correlate with the estimated results of the amount of EEG induction caused by sound stimulation (i.e., the degree to which gamma waves are induced), an index calculated from the degree of EEG synchronization at each time (i.e., the duration for which gamma wave synchronization was achieved), or an index calculated from the EEG induction ratio at each time (i.e., the history of the degree to which gamma waves are induced compared to other EEG bands). As one example, the score is determined to increase (e.g., linearly increase) with an increase in the amount of EEG induction. As another example, the score is determined to increase little when the amount of EEG induction is below a first threshold, and to increase with an increase in the amount of EEG induction after the amount of EEG induction exceeds the first threshold. As yet another example, the score is determined to increase with an increase in the amount of EEG induction when the amount of EEG induction is below a second threshold, and to increase little (i.e., saturate) after the amount of EEG induction exceeds the second threshold. By measuring the brain wave elicitation level of a subject when the subject listens to various acoustic signals under various conditions using an electroencephalograph, it is possible to describe the relationship between the elements included in the listening history and the brain wave elicitation level.Based on the relationship between the elements included in the listening history and the brain wave elicitation level and the relationship between the score and the brain wave elicitation level, it is possible to create a score function for deriving a score corresponding to the elements included in the listening history.The score function may be defined as a mathematical formula with the elements of the listening history as arguments, or may be defined as a lookup table.

[0082] As an example, the signal processing device 10 calculates a score corresponding to the listening history using the score function shown in Fig. 8. In the score function of Fig. 8, the score is defined so that it increases or remains the same (in other words, it does not decrease) as the listening period increases. Also, in the score function of Fig. 8, the score is defined so that the score for a playback volume C1 is higher than the score for a playback volume C2 (lower than C1).

[0083] When the signal processing device 10 acquires multiple listening histories in step S120, it may calculate the change in the cumulative score over the target period (i.e., the change in the cumulative score over time) by cumulatively adding up the scores corresponding to each listening history.

[0084] When a target score for the patient during the target period is set, the signal processing device 10 may calculate the ratio of the cumulative score to the target score as the patient's achievement level. The target score corresponds to a standard score (i.e., a sound that the patient should hear) that the patient should achieve during the target period in order to improve cognitive function. The signal processing device 10 may calculate the achievement level at a single point in time, or may calculate the change in the achievement level over the target period (i.e., the change in the achievement level over time).

[0085] The signal processing device 10 may store the score calculation results in a database (not shown), thereby enabling previously calculated scores to be obtained in a short time and with reduced consumption of computational resources.

[0086] After step S121, the signal processing device 10 executes notification of the target information (S122). Specifically, the signal processing device 10 transmits information necessary for presenting the target information generated in step S121 (for example, the target information itself, or information regarding the movement of light, images, sounds, characters, equipment, etc. that can be used to present the target information) to the related party terminal 70.

[0087] Information regarding the recipient of the target information (hereinafter referred to as "recipient information") is defined in association with information identifying the patient (e.g., patient ID). The recipient information may be the account name, email address, or telephone number of the relevant person to be notified. By using the recipient information, it is possible to identify the relevant person terminal 70 to which the information necessary for presenting the target information is to be sent.

[0088] The signal processing device 10 can notify the target information in at least one of the following formats, for example: Application notification SNS (Social Networking Service) message E-mail SMS (Short Service Message) message Information (e.g., a message) that can be viewed after logging in to a predetermined application or SaaS (Software as a Service)

[0089] The related party terminal 70 presents the target information to the related party via an output device. The signal processing device 10 can express the target information using at least one of light, sound, text, images, and the movement of equipment or the like.

[0090] As described above, the target information may include a score. The score included in the target information may be at least one of the following: - Score for each listening history - Cumulative score at any point in the target period - Changes in cumulative score during the target period - Achievement level at any point in the target period - Changes in achievement level during the target period - Degree of failure to achieve the target score

[0091] As an example, as shown in FIGS. 9, 10 and 11, the related party terminal 70 can present target information to the related party by an image displayed on the display 81.

[0092] When the related person has the authority to view the target information of multiple patients (typically, when the related person corresponds to a medical professional or a caregiver), the related person terminal 70 displays the screen P80 of Fig. 9 on the display 81. On the other hand, when the related person has the authority to view the target information of only one patient (typically, when the related person corresponds to the patient himself / herself or a family member of the patient), the related person terminal 70 displays the screen P81 of Fig. 9 on the display 81.

[0093] The screen P80 displays a list of patients whose target information can be viewed. As shown in Fig. 9, the screen P80 includes an object A801.

[0094] Object A801 ​​accepts a user instruction specifying which patient's target information to view. When object A801 ​​is selected (or the name is clicked), the related party terminal 70 displays screen P81 on the display 81.

[0095] Screen P81 is a menu screen for selecting which information about a patient to view. As shown in Fig. 9, screen P81 includes objects A811 to A812. Note that the selectable objects or display content on the menu screen may be limited depending on the authority of the person involved (e.g., whether or not they are a doctor).

[0096] The object A811 accepts a user instruction to display basic information about the patient. When the object A811 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the basic information about the patient.

[0097] The object A812 accepts a user instruction to display the patient's score, etc. When the object A812 is selected, the related party terminal 70 displays a screen P82 shown in FIG.

[0098] The object A813 accepts a user instruction to display the results of the patient's cognitive function test. When the object A811 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the results of the patient's cognitive function test.

[0099] The object A 814 accepts a user instruction to display the patient's medical record. When the object A 814 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the results of the patient's medical record.

[0100] Screen P82 is a dashboard screen related to the patient's hearing status. As shown in Fig. 10, screen P82 includes objects A821 to A827. Note that the objects that can be selected or the displayed content on the dashboard screen may be limited depending on the authority of the person involved (e.g., whether or not they are a doctor).

[0101] The object A821 displays the patient's achievement level for today (i.e., the viewing date of the screen P82). The object A821 represents the patient's achievement level using a pie chart and numbers.

[0102] The object A822 accepts a user instruction to display a detailed patient interview history. When the object A822 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the detailed patient interview history.

[0103] Object A823 displays the patient's past achievement history in the form of a bar graph. The achievement history displayed in object A823 can be switched between yearly, monthly, and weekly units. When a bar representing the achievement level of any day in object A823 is selected, the related party terminal 70 displays screen P83 of FIG. 11 on the display 81.

[0104] The object A 824 accepts a user instruction to display the history of doctor comments. When the object A 824 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the history of doctor comments.

[0105] The object A825 accepts a user instruction to display the results of the patient's cognitive function test. When the object A825 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the results of the patient's cognitive function test.

[0106] Object A826 accepts a user instruction to display statistical information. When object A826 is selected, the related party terminal 70 displays a screen (not shown) displaying statistical information on the display 81. Here, the statistical information corresponds to the result of statistical processing of listening histories or scores collected for multiple patients. The listening histories or scores of multiple patients may be collected by an external device (e.g., a server not shown) from the signal processing device 10 and other signal processing devices having similar functions to the signal processing device 10. The related party terminal 70 may display the statistical information in comparison with the listening histories or scores of the patients.

[0107] The object A 827 accepts a user instruction to display an editing screen for a message to be sent. When the object A 827 is selected, the related party terminal 70 displays an editing screen for a message to be sent (not shown) on the display 81.

[0108] The screen P83 displays the patient interview status on a specific date selected by the person concerned. As shown in Fig. 11, the screen P83 includes objects A831 to A834.

[0109] Object A831 displays the patient's achievement level for a selected specific date. Object A831 represents the patient's achievement level using a pie chart and numbers.

[0110] Object A832 uses a line graph to represent the progress of the patient's achievement level on a selected date. Object A832 clearly shows on the time axis the period in which the rate of increase in achievement level per unit time was high ("high efficiency"), the period in which the rate of increase in achievement level per unit time was low ("low efficiency"), the period in which processing (amplitude modulation) of the input audio signal was disabled ("processing OFF"), and the period in which playback of the output audio signal was stopped ("audio OFF"). The efficiency classification may be divided into three or more stages instead of two. The current time may also be displayed.

[0111] Object A833 accepts a user instruction to shift the selected date forward by one day. When object A833 is selected, the related party terminal 70 updates the contents of objects A831 to A832.

[0112] Object A834 accepts a user instruction to shift the selected date forward by one day. When object A834 is selected, the related party terminal 70 updates the contents of objects A831 to A832.

[0113] 9 to 11 show examples in which target information related to the listening status of a single selected patient is displayed. However, the present invention is not limited to this example. The signal processing device 10 may transmit target information related to multiple patients to the related party terminal 70, and the related party terminal 70 may simultaneously display the target information related to the multiple patients. For example, the related party terminal 70 may display information on the screen P82 related to each of the multiple patients side by side on the display 81. Furthermore, the number of information items, display size, or amount of information displayed on the display 81 may be changed depending on the number of patients for which information is to be displayed. This allows target information related to multiple patients to be efficiently displayed on a single screen.

[0114] The target information may be presented on an app screen or a website screen, or may be presented using a viewing screen for emails or other messages (e.g., SNS messages, SMS messages, etc.). Alternatively, the target information may be displayed on the screen of a smart speaker, a television, a smart watch, a car navigation system, etc.

[0115] Alternatively, the participant terminal 70 may represent the score by a sound output from a speaker (an example of an output device).

[0116] After step S122, the signal processing device 10 ends the information notification process of FIG.

[0117] (5) Summary As described above, the signal processing device 10 of this embodiment outputs an acoustic signal (i.e., an output acoustic signal) that is generated by amplitude modulation and has an amplitude change corresponding to the frequency of gamma waves, and stores information about the output acoustic signal (e.g., a listening history). The signal processing device 10 transmits a notification based on the stored information to a device external to the signal processing device 10 (i.e., the relevant person terminal 70). This allows the relevant person to understand the patient's listening status for sounds intended to improve cognitive function.

[0118] The signal processing device 10 may transmit a notification to at least one of a device owned by the patient (i.e., the user of the sound output device 30), a device owned by the patient's family, a device owned by the patient's caregiver, and a device owned by a medical professional. This allows at least one of the patient himself / herself, the patient's family, the patient's caregiver, and the medical professional to understand the patient's hearing status.

[0119] The information about the acoustic signal stored by the signal processing device 10 may include information about at least one of the period during which the acoustic signal was output, the modulation method applied to the acoustic signal, the loudness (volume) of the sound emitted in response to the acoustic signal, and the feature quantity of the acoustic signal, thereby enabling relevant parties to be notified of details of the patient's hearing condition.

[0120] The notification may include information indicating at least one of the listening history and the score determined based on the listening history, thereby enabling the relevant person to grasp the details of the listening situation of the patient.

[0121] The output acoustic signal may have amplitude variations corresponding to frequencies between 35 Hz and 45 Hz, inclusive, which may induce gamma waves in the patient's brain and improve cognitive function.

[0122] (6) Modifications A modification of this embodiment will be described.

[0123] (6-1) Modification 1 Modification 1 will be described below. Modification 1 is an example in which notification of target information is performed conditionally.

[0124] (6-1-1) Information Notification Processing The following describes the information notification processing of Modification 1. Fig. 12 is a flowchart of the information notification processing of Modification 1.

[0125] The information notification process of the first modification starts when the same start condition as that of the information notification process of the present embodiment is met.

[0126] 12, the signal processing device 10 acquires a listening history (S120) and generates target information (S121). Details of the acquisition of a listening history (S120) and the generation of target information (S121) are as described in the present embodiment.

[0127] After step S121, the signal processing device 10 executes a determination of the notification condition (S223). Specifically, the signal processing device 10 acquires information corresponding to the determination material and determines whether or not a predetermined notification condition is met based on the acquired information. The notification condition may be editable by the person concerned. The notification condition may be defined for each person concerned.

[0128] As a first example of determining whether the notification condition is met (S223), the signal processing device 10 may acquire a listening history collected about the patient and determine whether the acquired listening history satisfies a predetermined notification condition. For example, if no listening history has been recorded for even one day in the last week, or if a listening history has been recorded every day in the last three days, the signal processing device 10 may determine that the notification condition is met.

[0129] As a second example of determining whether the notification condition is met (S223), the signal processing device 10 may acquire a score calculated for the patient and determine whether the acquired score meets a predetermined notification condition. For example, if the average daily achievement level for the most recent week is below or above a threshold, the signal processing device 10 may determine that the notification condition is met.

[0130] As a third example of determining the notification condition (S223), the signal processing device 10 may acquire the results of a cognitive function test for the patient and determine whether the acquired results satisfy a predetermined notification condition. For example, if the results of the cognitive function test indicate a change (improvement or deterioration) in the patient's cognitive function class, the signal processing device 10 may determine that the notification condition is met. Note that the signal processing device 10 may acquire the results of the cognitive function test from an external device, or may acquire the results by administering the cognitive function test to the patient itself. In other words, the signal processing device 10 may be equipped with a function for conducting a cognitive function test (e.g., a function for measuring cognitive function through voice dialogue with the patient).

[0131] If it is determined in step S223 that the notification condition is met, the signal processing device 10 executes notification of the target information (S122). Details of the notification of the target information (S122) are as described in the present embodiment.

[0132] After step S122, the signal processing device 10 ends the information notification process in Fig. 12. Furthermore, if it is determined in step S223 that the notification condition is met, the signal processing device 10 ends the information notification process in Fig. 12.

[0133] (6-1-2) Summary As described above, the signal processing device 10 of the first modification conditionally notifies the target information. For example, the signal processing device 10 may transmit a notification when the listening history or the score meets a predetermined condition. This provides timely feedback to the relevant parties about the patient's signs of abandoning listening to the sounds or the patient's efforts, making it easier for the relevant parties to encourage the patient to continue listening to the sounds.

[0134] Alternatively, the signal processing device 10 may acquire the result of a cognitive function test of the patient and transmit a notification when the result of the cognitive function test satisfies a predetermined condition, thereby encouraging relevant parties to consider future actions for the patient (for example, setting a goal for listening to sounds to improve cognitive function) based on the result of the cognitive function test.

[0135] (6-2) Modification 2 Modification 2 will be described below. Modification 2 is an example in which target information is generated taking into consideration the patient's listening environment.

[0136] (6-2-1) Configuration of the Sound System The configuration of the sound system will be described below. Fig. 13 is a block diagram showing the configuration of the sound system of the second modification.

[0137] As shown in FIG. 13, the sound system 2 includes a signal processing device 10 , a sound output device 30 , a sound source device 50 , a related party terminal 70 , and a sensor 90 .

[0138] The signal processing device 10 and the sensor 90 may be connected to each other via a network (e.g., the Internet or an intranet) or via a predetermined interface (e.g., a wireless interface using Bluetooth or the like).

[0139] The sensor 90 senses the listening environment of the patient. Specifically, the sensor 90 measures physical quantities related to at least one of the following: the listening position of the patient (particularly, the relative position with respect to the sound output device 30); the sound pressure generated at the listening position of the patient by the sound emitted from the sound output device 30; and the patient's behavior when listening to the sound emitted from the sound output device 30 (for example, whether the patient is concentrating on listening or is paying attention to something else).

[0140] The sensor 90 may include at least one of the following, for example: - A device (for example, a device that transmits or receives radio signals) that constitutes a radio wave positioning system (for example, Bluetooth AoA (Angle of Arrival)) - A distance measurement sensor (for example, an ultrasonic sensor or LIDAR (Light Detection and Ranging)) - For example, a pressure sensor embedded in a chair, bed, floor, or the like - A camera (visible light camera or infrared camera) - A microphone

[0141] (6-2-2) Information Processing The information processing of the second modification will be described.

[0142] (6-2-2-1) Acoustic Signal Processing The following describes the acoustic signal processing of Modification 2. FIG.

[0143] The acoustic signal processing of the second modification example starts when the same start condition as that of the acoustic signal processing of this embodiment is met.

[0144] 14, the signal processing device 10 acquires an input acoustic signal (S110), generates an output acoustic signal (S111), and sends out the output acoustic signal (S112). The details of the acquisition of the input acoustic signal (S110), the generation of the output acoustic signal (S111), and the sending out of the output acoustic signal (S112) are as described in the present embodiment.

[0145] After step S212, the signal processing device 10 executes a determination of the listening environment (S214). Specifically, the signal processing device 10 acquires the sensing result of the sensor 90. The signal processing device 10 determines the listening environment of the patient based on the sensing result. The listening environment of the patient may include, for example, at least one of the following: - The distance from the sound output device 30 to the listening position of the patient, or a classification result thereof - The direction from the sound output device 30 to the listening position of the patient, or a classification result thereof - The sound pressure generated at the listening position of the patient by the sound emitted from the sound output device 30 (i.e., the degree to which the sound emitted from the sound output device 30 is attenuated), or a classification result thereof - The patient's behavior when listening to the sound emitted from the sound output device 30 (e.g., the type of activity, such as sitting, lying down, exercising, or operating a smartphone)

[0146] After step S214, the signal processing device 10 executes saving of the listening history (S113). Specifically, the signal processing device 10 saves the listening history including information related to the determination result in step S214 (hereinafter referred to as "listening environment information") in a listening history database. Other elements that can be included in the listening history are as described in the present embodiment.

[0147] The signal processing device 10 ends the audio signal processing of FIG. 14 at step S113. The signal processing device 10 may perform the audio signal processing of FIG. 14 all at once on an input audio signal having a certain playback period (e.g., one piece of music content), or may repeatedly perform the audio signal processing of FIG. 14 for each predetermined playback section of the input audio signal (e.g., every 100 ms). Alternatively, the signal processing device 10 may continuously perform modulation processing on the input audio signal, such as modulation using analog signal processing, and output a modulated audio signal. The audio signal processing shown in FIG. 14 may end in response to a specific end condition (e.g., a certain amount of time has elapsed, an operation has been performed by a related person, or the output history of modulated sound has reached a predetermined state).

[0148] (6-2-2-2) Information Notification Processing The information notification processing of the second modified example will be described.

[0149] The information notification process of the second modification starts when the same start condition as that of the information notification process of the present embodiment is met.

[0150] The signal processing device 10 executes the acquisition of the listening history (S120). The details of the acquisition of the listening history (S120) are as described in the present embodiment.

[0151] After step S120, the signal processing device 10 generates target information (S121). Specifically, the signal processing device 10 generates target information based on the listening history acquired in step S120. For example, the generated target information may include listening environment information or information obtained by processing the listening environment information (e.g., statistical information). Furthermore, similar to the present embodiment, the signal processing device 10 may calculate a score corresponding to the listening history using a score function. In the second modification, the signal processing device 10 may further correct the score corresponding to the listening history according to the listening environment information included in the listening history.

[0152] As a first example of generating target information (S121), the signal processing device 10 corrects the score to decrease or maintain the same as the distance from the sound output device 30 to the patient's listening position. As an example, assuming that the score before correction is the same, the signal processing device 10 corrects the score so that it is smaller when the distance is a first value than when the distance is a second value (the second value is smaller than the first value). At listening positions farther from the sound output device 30, there is a possibility that the sound pressure reaching the patient will decrease and the modulation degree will be relatively reduced due to room reverberation or external sounds, which may reduce the effect of listening to improve cognitive function. However, by performing such a correction, a more reasonable score can be obtained.

[0153] As a second example of generating target information (S121), the signal processing device 10 corrects the score to decrease or maintain its original value in response to an increase in the difference between the direction from the sound output device 30 to the patient's listening position and the direction of the sound output device 30's directivity (hereinafter referred to as the "directional difference"). As an example, assuming that the score before correction is the same, the signal processing device 10 corrects the score so that when the directional difference is a first value, the score is smaller than when the directional difference is a second value (the second value is smaller than the first value). The amount of correction may be determined according to the directional characteristics (gain characteristics) of the sound output device 30. If the listening position is located in a direction away from the directional direction of the sound output device 30, the sound pressure reaching the patient or the modulation level may decrease, potentially reducing the cognitive function improvement effect of listening. However, by performing such correction, a more appropriate score can be obtained.

[0154] As a third example of generating target information (S121), the signal processing device 10 corrects the score to decrease or maintain the score in response to a decrease in sound pressure or a decrease in modulation level caused at the patient's listening position by the sound emitted from the sound output device 30. As an example, assuming that the score before correction is the same, the signal processing device 10 corrects the score so that it is smaller when the sound pressure is a first value than when the sound pressure is a second value (the second value is greater than the first value). If a decrease in the sound pressure or modulation level reaching the patient occurs, the effect of improving cognitive function through listening may be reduced, but by performing such a correction, a more reasonable score can be obtained.

[0155] As a fourth example of generating target information (S121), a correction amount is determined in advance for each patient's behavior (e.g., listening situation, wakefulness state, stress level, etc.) when listening to the sound emitted from the sound output device 30, and the signal processing device 10 corrects the score according to the correction amount. Although the cognitive function improvement effect of listening to the sound emitted from the sound output device 30 may vary depending on the patient's behavior when listening to the sound emitted from the sound output device 30, performing such a correction makes it possible to obtain a more appropriate score.

[0156] After step S120, the signal processing device 10 executes notification of the target information (S122). Details of the notification of the target information (S122) are as described in the present embodiment.

[0157] (6-2-3) Summary As described above, the signal processing device 10 of Modification 2 may determine the patient's listening environment for the sound corresponding to the output acoustic signal and generate target information based on the determination result. This makes it possible to notify relevant parties of information about the patient's listening environment or notify relevant parties of a score calculated taking into account the influence of the patient's listening environment on the improvement effect of cognitive function.

[0158] (7) Other Modifications The storage device 11 may be connected to the signal processing device 10 via a network. The display 21 may be integrated with the signal processing device 10. The storage device 71 may be connected to the related party terminal 70 via a network. The display 81 may be integrated with the related party terminal 70.

[0159] Each step of the above information processing can be executed by either the signal processing device 10 or the related party terminal 70. Some of the steps of the above information processing may be executed by the related party terminal 70 or an external device (e.g., a server) not shown. For example, the signal processing device 10 may store a listening history in the external device, the external device may generate target information based on the listening history, and the signal processing device 10 may acquire and notify the target information generated by the external device. In this case, the signal processing device 10 may omit the process of S120 in FIG. 7 and acquire a score from the external device in S121.

[0160] The above description shows an example in which the signal processing device 10 is connected to one sound output device 30. The signal processing device 10 may be connected to a plurality of sound output devices 30, and in this case, the signal processing device 10 may be configured to be able to select to which sound output device 30 an output sound signal is to be sent.

[0161] The above description has shown an example in which the signal processing device 10 is connected to one sound source device 50. The signal processing device 10 may be connected to a plurality of sound source devices 50, and in this case, the signal processing device 10 may be configured to be able to select from which sound source device 50 the input acoustic signal is to be acquired.

[0162] The above description focuses on the case where one signal processing device 10 is associated with one patient. However, this is not limiting, and multiple patients may be associated with one signal processing device 10. For example, consider a case where multiple patients (users listening to the sound output from the sound output device 30) are present in a room where the signal processing device 10 and a speaker serving as the sound output device 30 are installed. In this case, the signal processing device 10 identifies the multiple patients present in the room and updates the listening history database corresponding to each patient based on the output result of the sound signal from the signal processing device 10. This makes it possible to easily grasp the listening status of each patient even when multiple patients listen to the sound based on the sound signal output from a single signal processing device 10. One method for the signal processing device 10 to identify patients listening to the sound emitted from the sound output device 30 (e.g., patients in the same room as the sound output device 30) is to obtain registered ID information when the patient enters the room. Another method is to identify patients near the sound output device 30 by wireless communication, such as Bluetooth, between a terminal device owned by the patient and the sound output device 30. However, the present invention is not limited to these methods, and the signal processing device 10 can use various existing identification methods.

[0163] Furthermore, multiple signal processing devices 10 may be associated with one patient. For example, consider a case where a signal processing device 10 and a speaker serving as an audio output device 30 are installed in each of multiple rooms in a facility where patients reside. In this case, the multiple signal processing devices 10 in the facility access a common listening history database. Each signal processing device 10 identifies the patient in the same room as the audio output device 30 connected to that signal processing device 10, and updates the listening history database corresponding to that patient based on the output result of the audio signal from that signal processing device 10. This makes it possible to easily grasp the listening status of a single patient even when the patient listens to sounds based on audio signals output from multiple signal processing devices 10. The signal processing device 10 can use the various methods described above to identify the patient.

[0164] Similarly, a plurality of signal processing devices 10 may be associated with a plurality of patients. That is, each of the plurality of signal processing devices 10 may update a listening history database for each of the plurality of patients who listen to sounds based on acoustic signals output from the signal processing device 10. Then, the signal processing device 10 may transmit a notification based on information acquired from the listening history database to the related party terminal 70 associated with each patient.

[0165] The signal processing device 10 may be equipped with a function for conducting a cognitive function test (for example, a function for measuring cognitive function through voice dialogue with a patient). In this case, the signal processing device 10 may notify the relevant party terminal 70 of target information including information indicating at least one of the patient's answers to the cognitive function test and the results of the cognitive function test. This allows the relevant party to timely grasp the patient's hearing status as well as the patient's cognitive function.

[0166] In the second modification, an example of calculating a score taking into account listening environment information was shown. The signal processing device 10 may also calculate a score taking into account other information. As a first example, the signal processing device 10 may acquire information regarding the attributes of a patient listening to a sound corresponding to an output acoustic signal (hereinafter referred to as "attribute information") and calculate a score based on the attribute information. The attribute information may include, for example, at least one of the patient's age, gender, hearing ability (which may include hearing ability by frequency band or speech audibility), dementia risk, past listening history (which may include analysis results of the past listening history), preferences or intentions (commitment) regarding listening to sounds, and results of past electroencephalogram (EEG) measurements or psychological experiments, or feature quantities based on those results. This allows for a score that takes into account the influence of the patient's attributes on the improvement of cognitive function. The signal processing device 10 may use a sensor 90 (particularly, a device constituting a radio wave positioning system or a camera) to identify the patient. A hearing test function may be implemented in the signal processing device 10 or the related party terminal 70 to measure the patient's hearing ability. As a second example, the signal processing device 10 may acquire the results of a cognitive function test of a patient who listens to a sound corresponding to the output acoustic signal (i.e., the patient's current cognitive function), and calculate a score based on the test results. This makes it possible to obtain a score that takes into account the influence of the patient's current cognitive function on the improvement effect of cognitive function. The signal processing device 10 or the related party terminal 70 may be equipped with a function for conducting a cognitive function test (e.g., a function for measuring cognitive function through voice dialogue with the patient).

[0167] The signal processing device 10 may change the method for generating an output audio signal (e.g., modulation method, synthesis ratio of two or more audio signals, method for determining an audio signal to which amplitude modulation is applied, etc.) or the playback volume of a sound corresponding to the output audio signal, depending on the patient's score. As a first example, the signal processing device 10 may generate an output audio signal corresponding to a sound that will result in a higher score being attained by listening, or may increase the playback volume, the shorter the time remaining until the expiration of the target score-setting period. As a second example, the signal processing device 10 may generate an output audio signal corresponding to a sound that will result in a higher score being attained by listening, or may increase the playback volume, the lower the patient's current achievement level. Furthermore, if the score achievement level does not meet a predetermined standard, the signal processing device 10 may automatically play an amplitude-modulated audio signal or may play a predetermined audio signal that has a high brainwave elicitation effect (e.g., a beep or chirp sound unrelated to the input audio signal and having a frequency corresponding to gamma waves). This makes it possible to assist the patient in achieving the target score without paying attention to their score achievement status.

[0168] In the above description, an example has been given in which the score is calculated based on the listening history stored by the signal processing device 10. However, in addition to or instead of the listening history, the score may be calculated based on an audio signal received by a microphone (e.g., the intensity, duration, waveform, or a combination thereof). Such a microphone may be installed closer to the patient's listening position than the sound output device 30, and may be, for example, a microphone included in a terminal used by the patient, or a microphone that can be worn on the patient's body (e.g., near the ear).

[0169] An algorithm for determining a method for generating an output audio signal (e.g., a modulation method, a combining ratio of two or more audio signals, a method for determining an audio signal to which amplitude modulation is applied, etc.), or a default method for generating an output audio signal, may be changed by updating the firmware of the signal processing device 10. The firmware update is performed, for example, by the signal processing device 10 communicating with an external device (e.g., a server not shown). Similarly, a method for generating target information (which may include a method for calculating a score), a method for notifying the target information, a method for presenting the target information, etc. may also be changed by updating the firmware of the signal processing device 10. The contents of the firmware update of the signal processing device 10 may be notified to relevant parties in advance, and the signal processing device 10 may provide an operating mode that allows relevant parties to compare sounds before and after the update.

[0170] Although an example in which the input acoustic signal is acquired from the sound source device 50 has been shown, the input acoustic signal may be generated by receiving environmental sound with a microphone. The input acoustic signal may be read from the storage device 11 or a removable medium connected to the signal processing device 10. The sound source device 50 (including the sound source device 50 built into the signal processing device 10) may generate the input acoustic signal each time.

[0171] Although the above description has been given of an example in which an audio signal derived from an input audio signal is modulated, an output audio signal may also be generated based on the input audio signal and a modulated audio signal that is not derived from the input audio signal.

[0172] The above description has focused on an example in which the modulation function has a frequency of 35 Hz or more and 45 Hz or less. However, the modulation function used by the signal processing device 10 is not limited to this, and any modulation function that affects the induction of gamma waves in the listener's brain may be used. For example, the modulation function may have a frequency of 25 Hz or more and 140 Hz or less. Furthermore, for example, the frequency of the modulation function may change over time, and the modulation function may partially have a frequency below 35 Hz or a frequency higher than 45 Hz.

[0173] In the above description, the output acoustic signal generated by the signal processing device 10 is output to the acoustic output device 30, which generates sound and allows a patient to hear it. However, the output destination of the output acoustic signal from the signal processing device 10 is not limited to this. For example, the signal processing device 10 may output the output acoustic signal to an external storage device or information processing device via a communication network or by broadcasting. In this case, the signal processing device 10 may output an unmodulated input acoustic signal to the external device along with the output acoustic signal generated by modulation. This allows the external device to arbitrarily select and play either the unmodulated acoustic signal or the modulated acoustic signal. The signal processing device 10 may also output information indicating the details of the modulation process to the external device along with the output acoustic signal. The information indicating the details of the modulation process may include, for example, any of the following: Information indicating a sound source corresponding to the modulated acoustic signal; Information indicating a channel corresponding to the modulated acoustic signal; Information indicating the characteristics of the modulated acoustic signal; Information indicating a modulation function; Information indicating a modulation depth; and Information indicating a volume. This allows the external device to change the playback method of the acoustic signal depending on the details of the modulation process.

[0174] In the above description, an example has been shown in which the signal processing device 10 performs modulation processing including amplitude modulation to generate a modulated acoustic signal. However, the signal processing device 10 may also acquire a modulated acoustic signal generated by an external device and transmit an output acoustic signal based on the modulated acoustic signal. In this case, the signal processing device 10 may acquire information necessary for generating a listening history (e.g., modulation method information or feature information) from the external device.

[0175] The related party terminal 70 may collect other information (e.g., TV viewing status (type of program, viewing time, etc.), patient vital data obtained from a wearable terminal) and present the collected information together with the target information.

[0176] Furthermore, when the signal processing device 10 acquires additional information (for example, an ID3 tag in an MP3 file) along with the input audio signal, the signal processing device 10 may modify the additional information and output it to an external device together with the output audio signal.

[0177] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the above-described embodiments. Furthermore, the above-described embodiments can be improved or modified in various ways without departing from the spirit of the present invention. Furthermore, the above-described embodiments and modifications can be combined.

[0178] 1: Acoustic system 2: Acoustic system 10: Signal processing device 11: Storage device 12: Processor 13: Input / output interface 14: Communication interface 21: Display 30: Sound output device 50: Sound source device 70: Person involved terminal 71: Storage device 72: Processor 73: Input / output interface 74: Communication interface 81: Display 90: Sensor

Claims

1. an output means for outputting from the signal processing device an acoustic signal generated by amplitude modulation, the acoustic signal having an amplitude change corresponding to a frequency of gamma waves; a storage means for storing information on the acoustic signal output by the output means; a transmitting means for transmitting a notification based on the information stored in the storage means to a device external to the signal processing device; A system having

2. The system of claim 1, wherein the transmitting means transmits the notification to at least one of a device owned by a user who listens to the acoustic signal output by the output means, a device owned by a family member of the user, a device owned by a caregiver of the user, and a device owned by a medical professional.

3. 2. The system according to claim 1, wherein the information on the acoustic signal stored in the memory means includes information indicating at least one of a period during which the acoustic signal was output by the output means, a modulation method for the output acoustic signal, a magnitude of the output acoustic signal, and a feature of the output acoustic signal.

4. The system according to claim 1 , wherein the notification transmitted by the transmitting means includes information indicating at least one of an output history of the acoustic signal by the output means and a score determined based on the output history of the acoustic signal.

5. The system according to claim 1 , wherein the sending means sends the notification in response to at least one of the output history of the acoustic signal by the output means and a score determined based on the output history of the acoustic signal satisfying a predetermined condition.

6. A means for acquiring the results of a cognitive function test is provided; The system according to claim 1 , wherein the sending means sends the notification in response to the result of the cognitive function test acquired by the acquiring means satisfying a predetermined condition.

7. The system according to claim 1 , wherein the transmission means transmits, together with the notification, information indicating at least one of answers to a cognitive function test and a result of the cognitive function test.

8. 2. The system of claim 1, wherein the acoustic signal output by said output means has amplitude variations corresponding to frequencies between 35 Hz and 45 Hz.

9. The system according to claim 1 , further comprising an audio output device that allows a user to hear a sound corresponding to the audio signal output by the signal processing device.

10. 1. A method performed by a system including a signal processing device and an external device, comprising: outputting from the signal processing device an acoustic signal generated by amplitude modulation, the acoustic signal having an amplitude change corresponding to a frequency of gamma waves; storing information about the output acoustic signal; sending a notification to the external device based on the stored information; method.

11. A program for causing a computer to function as each of the means of the system according to any one of claims 1 to 8.