Signal processing device, cognitive function improvement system, signal processing method, and program

Abstract

Provided are a signal processing device, a cognitive function improvement system, a signal processing method, and a program that allow a person involved to grasp a user's listening situation for a sound corresponding to an amplitude-modulated acoustic signal.SOLUTION: In the sound system, the signal processing device 10 includes an output unit that outputs, to the sound output device 30, a sound signal that is generated by amplitude modulation and has a change in amplitude corresponding to a frequency of gamma waves, a storage unit that stores information of the sound signal output from the output unit in a storage device, and a transmission unit that transmits a notification based on the information stored by the storage unit to the concerned party terminal 70 that is a device outside the signal processing device.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a signal processing device, a cognitive function improvement system, a signal processing method, and a program. [Background technology]

[0002] Research has shown that inducing gamma waves in the brain of an organism by exposing it to pulsed sound stimuli at a frequency of about 40 times per second is effective in improving the organism's cognitive function (see Non-Patent Document 1). Gamma waves refer to neural oscillations that capture periodic neural activity in the brain's cortex using electrophysiological techniques such as electroencephalography and magnetoencephalography, and whose frequencies fall within the gamma band (25-140 Hz).

[0003] Patent document 1 discloses adjusting the volume by increasing or decreasing the amplitude of a sound wave or sound track to create a rhythmic stimulus corresponding to a stimulus frequency that induces brain wave entrainment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2020-501853 [Non-patent literature]

[0005] [Non-Patent Document 1] Multi-sensory Gamma Stimulation Ameliorates Alzheimer's-Associated Pathology and Improves Cognition Cell 2019 Apr 4;177(2):256-271.e22. doi: 10.1016 / j.cell.2019.02.014. Summary of the Invention [Problem to be solved by the invention]

[0006] When a user listening to sounds using amplitude-modulated acoustic signals in the hope of improving cognitive function is required, relevant parties may want to understand the user's listening status. For example, the user's family may feel uneasy if they cannot confirm that the user is listening to the sounds. Medical professionals who administer treatment through listening to sounds cannot provide appropriate treatment instructions unless they can confirm whether the user (patient) is listening to the sounds. If the user does not have an opportunity to review their listening status if they forget to listen to the sounds, it may be difficult for them to make listening to the sounds a habit.

[0007] An object of the present disclosure is to provide a technology that enables related parties to understand the listening conditions of a user with respect to a sound corresponding to an amplitude-modulated acoustic signal. [Means for solving the problem]

[0008] A signal processing device according to one aspect of the present disclosure includes an output means for outputting an acoustic signal generated by amplitude modulation, the acoustic signal having an amplitude change corresponding to the frequency of gamma waves, a memory means for storing information about the acoustic signal output by the output means, and a transmission means for transmitting a notification based on the information stored by the memory means to a device external to the signal processing device. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing the configuration of an audio system according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing a configuration of a signal processing device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a block diagram showing the configuration of a related party terminal according to the present embodiment. [Figure 4] FIG. 1 is an explanatory diagram of one aspect of the present embodiment. [Figure 5] FIG. 2 is a diagram showing the data structure of a listening history database according to the present embodiment. [Figure 6] 3 is a flowchart of acoustic signal processing according to the present embodiment. [Figure 7]10 is a flowchart of an information notification process according to the present embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a score function used in the information notification process of the present embodiment. [Figure 9] 10A and 10B are diagrams illustrating an example of presentation of target information in the information notification process of the present embodiment. [Figure 10] 10A and 10B are diagrams illustrating an example of presentation of target information in the information notification process of the present embodiment. [Figure 11] 10A and 10B are diagrams illustrating an example of presentation of target information in the information notification process of the present embodiment. [Figure 12] 10 is a flowchart of an information notification process according to the first modification. [Figure 13] FIG. 10 is a block diagram showing the configuration of an audio system according to a second modification. [Figure 14] 10 is a flowchart of acoustic signal processing according to Modification 2. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the drawings for explaining the embodiment, the same components are generally designated by the same reference numerals, and repeated description thereof will be omitted.

[0011] (1) Sound system configuration The configuration of the audio system will now be described. Fig. 1 is a block diagram showing the configuration of the audio system of this embodiment.

[0012] As shown in FIG. 1, the audio system 1 includes a signal processing device 10, an audio output device 30, a sound source device 50, and a related party terminal .

[0013] The signal processing device 10 and the sound source device 50 are connected to each other via a predetermined interface capable of transmitting an acoustic signal. The interface is, for example, SPDIF (Sony Philips Digital Interface), HDMI (registered trademark), High-Definition Multimedia Interface), a pin connector (RCA pin), or an audio interface for headphones. The interface may also be a wireless interface using Bluetooth (registered trademark), etc. The signal processing device 10 and the acoustic output device 30 are similarly connected to each other via a predetermined interface. The acoustic signal in this embodiment includes either or both of an analog signal and a digital signal.

[0014] The signal processing device 10 and the related party terminal 70 may be connected to each other via a network (for example, the Internet or an intranet), or may be connected to each other via a predetermined interface (for example, a wireless interface using Bluetooth, etc.). Note that the signal processing device 10 and the related party terminal 70 may also be connected to each other via an external device (for example, a server) not shown.

[0015] The signal processing device 10 performs acoustic signal processing on an input acoustic signal acquired from a sound source device 50. The acoustic signal processing by the signal processing device 10 includes at least modulation processing of the acoustic signal (details will be described later). The acoustic signal processing by the signal processing device 10 may also include conversion processing of the acoustic signal (for example, separation, extraction, or synthesis). Furthermore, the acoustic signal processing by the signal processing device 10 may further include amplification processing of the acoustic signal similar to that of, for example, an AV amplifier. The signal processing device 10 sends an output acoustic signal generated by the acoustic signal processing to the acoustic output device 30. The signal processing device 10 is an example of an information processing device.

[0016] The sound output device 30 generates sound according to the output sound signal acquired from the signal processing device 10. The sound output device 30 is, for example, a loudspeaker (which may include a speaker with a built-in amplifier (a powered speaker)), headphones, earphones, or a smart speaker. The sound output device 30 can also be configured as a single device together with the signal processing device 10. Specifically, the signal processing device 10 and the sound output device 30 can be implemented in a TV, radio, music player, AV amplifier, speaker, headphones, earphones, smartphone, PC (including tablet PC; the same applies below), or smart speaker. The signal processing device 10 and the sound output device 30 configure a cognitive function improvement system.

[0017] The sound source device 50 transmits an input acoustic signal to the signal processing device 10. The sound source device 50 is, for example, a TV, a radio, a music player, a smartphone, a PC, an electronic musical instrument, a telephone, a game console, an amusement machine, or a device that transmits an acoustic signal via broadcasting or information communication.

[0018] The related party terminal 70 can be implemented as a smartphone or a PC. The related party terminal 70 has a function of receiving information from the signal processing device 10 and presenting the notified information to the user (related party) of the related party terminal 70.

[0019] In the following description, a related person is, for example, at least one of the following people: A user of the sound output device 30 (i.e., a person who listens to sounds to improve cognitive function, hereinafter referred to as a "patient") Family members, friends, or acquaintances of the patient Healthcare professionals (e.g., a patient's doctor, nurse, or medical technician) Caregivers (e.g., the patient's caregiver) Creator or provider of the content corresponding to the input audio signal (described below) Creator or provider of the content corresponding to the input audio signal Provider of the signal processing device 10 -Managers of facilities used by patients Insurance companies Other patients in the same community as the patient System provider Third parties designated by the parties listed above The related party terminal 70 is owned by the related party. Here, "ownership" is not limited to ownership, but may also mean temporary exclusive use for operation. The related party terminal 70 may be shared by multiple related parties. Furthermore, the signal processing device 10 may notify information to multiple related party terminals 70.

[0020] (1-1) Configuration of signal processing device The configuration of the signal processing device will now be described with reference to Fig. 2, which is a block diagram showing the configuration of the signal processing device of this embodiment.

[0021] 2, the signal processing device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14. The signal processing device 10 is connected to a display 21.

[0022] The storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of a read-only memory (ROM), a random access memory (RAM), and storage (for example, a flash memory or a hard disk). The programs and data may be provided via a network, or may be provided by being recorded on a computer-readable recording medium.

[0023] The programs include, for example, the following programs: OS (Operating System) programs Application programs that perform information processing

[0024] The data includes, for example, the following data: Databases referenced in information processing Data obtained by performing information processing (i.e., the results of performing information processing)

[0025] The processor 12 is a computer that realizes the functions of the signal processing device 10 by reading and executing a program stored in the storage device 11. Note that at least a part of the functions of the signal processing device 10 may be realized by one or more dedicated circuits. The processor 12 is, for example, at least one of the following: ·CPU(Central Processing Unit) ·GPU(Graphic Processing Unit) ·ASIC(Application Specific Integrated Circuit) ·FPGA(Field Programmable Array) ·DSP(Digital Signal Processor)

[0026] The input / output interface 13 is configured to obtain information (e.g., user instructions) from an input device connected to the signal processing device 10, and to output information (e.g., an image or a control signal) to an output device connected to the signal processing device 10. The input device is, for example, a sound source device 50, a physical button, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, the display 21, the audio output device 30, a light-emitting element, a speaker different from the audio output device 30, or a combination thereof.

[0027] Furthermore, the input / output interface 13 may include signal processing hardware such as, for example, an A / D converter, a D / A converter, an amplifier, a mixer, a filter, and the like.

[0028] The communication interface 14 is configured to control communication between the signal processing device 10 and an external device (for example, the sound output device 30, the sound source device 50, or the participant terminal 70).

[0029] The display 21 is configured to display an image (still image or moving image). The display 21 is, for example, a liquid crystal display or an organic EL display. Alternatively, the display 21 may be a seven-segment display.

[0030] (1-2) Configuration of the relevant terminal The configuration of the related party terminal will now be described with reference to Fig. 3, which is a block diagram showing the configuration of the related party terminal of this embodiment.

[0031] 3, the related party terminal 70 includes a storage device 71, a processor 72, an input / output interface 73, and a communication interface 74. The related party terminal 70 is connected to a display 81.

[0032] The storage device 71 is configured to store programs and data. The storage device 71 is, for example, a combination of ROM, RAM, and storage (for example, flash memory or a hard disk). The programs and data may be provided via a network, or may be provided by being recorded on a computer-readable recording medium.

[0033] The programs include, for example, the following programs: OS programs Application programs that perform information processing

[0034] The data includes, for example, the following data: Databases referenced in information processing - Results of information processing

[0035] The processor 72 is a computer that realizes the functions of the related party terminal 70 by reading and executing the programs stored in the storage device 71. Note that at least a part of the functions of the related party terminal 70 may be realized by one or more dedicated circuits. The processor 72 is, for example, at least one of the following: ·CPU GPU ASIC FPGA DSP

[0036] The input / output interface 73 is configured to acquire information (for example, a user's instruction) from an input device connected to the related party terminal 70, and to output information (for example, an image) to an output device connected to the related party terminal 70. The input device is, for example, a physical button, a keyboard, a pointing device, a touch panel, or a combination thereof. The output device is, for example, a display 81, a speaker, or a combination thereof.

[0037] The communication interface 74 is configured to control communication between the participant terminal 70 and an external device (for example, the signal processing device 10).

[0038] The display 81 is configured to display an image (a still image or a moving image). The display 81 is, for example, a liquid crystal display or an organic EL display.

[0039] (2) One aspect of the embodiment An example of this embodiment will now be described with reference to Fig. 4, which is an explanatory diagram of this example.

[0040] As shown in FIG. 4, the signal processing device 10 acquires an acoustic signal (input acoustic signal) from the sound source device 50. The signal processing device 10 performs amplitude modulation processing on the acoustic signal. The signal processing device 10 outputs the generated acoustic signal (output acoustic signal) to the acoustic output device 30. Since the output acoustic signal has amplitude changes corresponding to the frequency of gamma waves, when the patient PA1 listens to the sound generated by the output acoustic signal, gamma waves are induced in the brain of the patient PA1 (brain waves are synchronized with the gamma frequency). This is expected to improve cognitive function (for example, to treat or prevent dementia). Meanwhile, the signal processing device 10 stores the patient PA1's listening history (an example of "information on the output acoustic signal") of the sound for improving cognitive function (the sound generated by the acoustic output device 30 in response to the output acoustic signal). As an example, the signal processing device 10 stores information in a listening history database (described later) regarding what kind of amplitude modulation was performed on what kind of feature-containing acoustic signal to generate the output acoustic signal, as well as information regarding the playback volume and duration at which the patient PA1 listened to the sound from the output acoustic signal.

[0041] The signal processing device 10 generates information (hereinafter referred to as "target information") regarding the patient PA1's hearing status for sounds for improving cognitive function based on the stored hearing history, and notifies the relevant party terminal 70 of the target information. The relevant party terminal 70 presents the target information to the relevant party PC2. This allows the relevant party PC2 to understand the patient PA1's hearing status for sounds.

[0042] This allows the person concerned PC2 to obtain various effects according to the attributes of the person concerned PC2. For example, a family member or a medical professional can encourage the patient PA1 to improve his / her hearing condition based on the notified target information.

[0043] Based on the notified target information, medical professionals can confirm whether patient PA1 is acting in accordance with the treatment plan, create or modify a treatment plan for patient PA1's cognitive function based on the listening situation, and make a diagnosis of patient PA1's cognitive function based on the listening situation. The patient PA1 can introspect on his / her own hearing situation based on the notified target information.

[0044] The facility manager can use the notified target information to efficiently manage the patient PA1 as a facility user. The insurance company can use the notified target information to calculate a reasonable insurance premium for patient PA1.

[0045] The provider of the signal processing device 10 can use the notified target information to conduct research and development to improve the cognitive function improvement effect of the signal processing device 10 (for example, a more effective method of generating and providing an output acoustic signal). The target information notified to the concerned person's PC 2 may be anonymized. Also, the patient may be allowed to freely choose whether or not to provide the target information in exchange for compensation (for example, a coupon).

[0046] The creator or provider of the content corresponding to the input acoustic signal can obtain suggestions for guidelines on creating or providing the content from the notified target information. The target information notified to the concerned person PC2 may be anonymized. Also, the patient may be free to choose whether or not to provide the target information in exchange for compensation (e.g., a coupon).

[0047] By viewing the notified target information, other patients belonging to the same community can check the behavioral records of their rival or fellow patients and reflect on their own behavioral records, making it easier to maintain motivation to continue listening to sounds to improve dementia.In addition, patients are aware that their behavioral records are being seen by other patients belonging to the same community, making it easier to maintain motivation to continue listening to sounds to improve dementia.

[0048] (3) Database The databases of this embodiment will be described below. The following databases are stored in the storage device 11. Alternatively, the databases may be stored in a storage device provided in an external device (e.g., a server) accessible by the signal processing device 10. In this case, information relating to a plurality of different signal processing devices 10 may be aggregated in one database.

[0049] (3-1) Listening history database The listening history database of this embodiment will now be described with reference to Fig. 5, which shows the data structure of the listening history database of this embodiment.

[0050] The listening history database stores listening history, which is log information about the patient's listening behavior to sounds for improving cognitive function.

[0051] 5, the listening history database includes an "ID" field, a "start date and time" field, an "end date and time" field, a "modulation method" field, a "feature value" field, and a "playback volume" field. Each field is associated with another field.

[0052] The "ID" field stores a listening history ID, which is information for identifying a listening history.

[0053] The "start date and time" field stores start date and time information, which is information about the date and time when the reproduction of the output acoustic signal started in the corresponding listening history.

[0054] The "End Date and Time" field stores end date and time information. The end date and time information is information about the date and time when playback of the output acoustic signal ended in the corresponding listening history. In other words, the period during which the patient listened to the sound of the output acoustic signal can be identified based on the start date and time information and the end date and time information.

[0055] The "Modulation Method" field stores modulation method information. The modulation method information is information about the processing performed to generate an output acoustic signal from an input acoustic signal in the corresponding listening history. For example, the modulation method information may include information about at least one of the following: Modulation function used for amplitude modulation Modulation depth Amplitude-modulated acoustic signal -Audio signal with no amplitude modulation applied Here, when an intermediate acoustic signal (described in detail below) including multiple acoustic signals is generated based on an input acoustic signal, and whether or not to apply amplitude modulation is determined individually for each of the multiple acoustic signals, each of the multiple acoustic signals corresponds to either an "acoustic signal to which amplitude modulation has been applied" or an "acoustic signal to which amplitude modulation has not been applied." Similarly, when an input acoustic signal is made up of multiple acoustic signals, and whether or not to apply amplitude modulation is determined individually for each of the multiple acoustic signals, each of the multiple acoustic signals corresponds to either an "acoustic signal to which amplitude modulation has been applied" or an "acoustic signal to which amplitude modulation has not been applied." In these cases, the function used for amplitude modulation or the modulation depth may differ for each of the amplitude-modulated acoustic signals, and therefore the modulation method information may include information on the "function used for amplitude modulation" or "modulation depth" of each acoustic signal.

[0056] The "feature" field stores feature information. The feature information is information about the feature of the acoustic signal to which amplitude modulation has been applied in the corresponding listening history. For example, the feature information may include information about at least one of the following: Distribution of amplitude, energy or rms value in the frequency domain (e.g., the result of a fast Fourier transform (FFT)) Distribution of amplitude, energy or RMS value in the time domain Type of sound source (e.g. vocals, instruments, objects, music, dialogue, natural sounds, or electronic sounds) Content attributes (e.g., program information for TV programs that contain audio signals)

[0057] The "playback volume" field stores playback volume information. The playback volume information is information related to the playback volume of the output audio signal in the corresponding listening history. The signal processing device 10 may control the playback volume information by specifying the playback volume to the audio output device 30, or may obtain the playback volume information from the audio output device 30.

[0058] (4) Information processing The information processing of this embodiment will be described.

[0059] (4-1) Acoustic signal processing The acoustic signal processing of this embodiment will be described below with reference to the flowchart of FIG.

[0060] The acoustic signal processing in FIG. 6 is realized by the processor 12 of the signal processing device 10 reading and executing a program stored in the storage device 11.

[0061] The acoustic signal processing in FIG. 6 starts when any of the following start conditions is met. The acoustic signal processing in Figure 6 was called by another process or an external instruction. The person involved performed the operation to call up the acoustic signal processing in Figure 6. The signal processing device 10 is in a predetermined state (for example, power is turned on). The appointed date and time has arrived. A predetermined time has elapsed since a predetermined event (for example, the start-up of the signal processing device 10 or the previous execution of the acoustic signal processing in FIG. 6).

[0062] As shown in FIG. 6, the signal processing device 10 acquires an input acoustic signal (S110). Specifically, the signal processing device 10 receives an input acoustic signal sent from a sound source device 50 . In step S110, the signal processing device 10 may further perform A / D conversion of the input acoustic signal.

[0063] The input sound signal may correspond to at least one of the following: Musical content (e.g., singing, playing, or a combination thereof (i.e., music). This may include audio content accompanying video content.) Audio content (e.g., readings, narrations, announcements, radio plays, solo performances, conversations, monologues, or any combination thereof. This may include audio content accompanying video content.) Other audio content (e.g., electronic, environmental, or mechanical sounds) However, singing or audio content is not limited to sounds produced by the human vocal tract, but may also include sounds generated by voice synthesis technology.

[0064] After step S110, the signal processing device 10 generates an output acoustic signal (S111). Specifically, the signal processing device 10 generates an output acoustic signal by amplitude modulating at least a part of the input acoustic signal acquired in step S110.

[0065] As a first example of generating an output acoustic signal (S111), the signal processing device 10 generates an intermediate acoustic signal including a plurality of acoustic signals having different characteristics based on the input acoustic signal. Here, the characteristics may be determined based on an input operation by a person involved or an external instruction, or may be determined by an algorithm. For example, the signal processing device 10 may determine the characteristics for generating the intermediate acoustic signal based on the results of analyzing the input acoustic signal.

[0066] The characteristics may be, for example, at least one of the following: ·Sound characteristics (especially qualitative characteristics) Frequency characteristics Time characteristics Amplitude characteristics Output characteristics

[0067] The signal processing device 10 selects one or more acoustic signals (hereinafter referred to as "target signals") to which amplitude modulation is to be applied from the multiple acoustic signals included in the intermediate acoustic signal. Which acoustic signal is selected as the target signal may be determined based on an input operation by a person involved, an external instruction, or an algorithm. For example, the signal processing device 10 may determine the target signal based on characteristics of the acoustic signal (balance between voice and music, volume change, type of music, timbre, or other characteristics). This allows the signal processing device 10 to select a target signal that will enhance the effect of modulation on improving cognitive function or that will minimize discomfort to the patient. However, the signal processing device 10 may also treat all of the multiple acoustic signals included in the intermediate acoustic signal as target signals.

[0068] The signal processing device 10 performs amplitude modulation on the selected target signal. As an example, the signal processing device 10 performs amplitude modulation on the target signal using a modulation function having a frequency corresponding to gamma waves (for example, a frequency of 35 Hz or more and 45 Hz or less). Specifically, assuming that a modulation function having a periodicity of 35 Hz or more and 45 Hz or less is A(t), a function representing the waveform of the acoustic signal before modulation is X(t), and a function representing the waveform of the modulated acoustic signal is Y(t), Y(t)=A(t)·X(t) As a result, a change in amplitude corresponding to the frequency is added to the target signal. Here, the signal processing device 10 may determine a modulation function used for amplitude modulation of the target signal or a modulation depth for the amplitude modulation. The modulation function or modulation depth may be determined based on an input operation by a related person or an external instruction, or may be determined by an algorithm. Furthermore, the signal processing device 10 may determine a common modulation function or modulation depth for multiple target signals, or may determine a modulation function or modulation depth individually for multiple target signals.

[0069] The signal processing device 10 generates an output acoustic signal based on an acoustic signal (hereinafter referred to as a "non-target signal") that is not selected as a target signal from among multiple acoustic signals included in the intermediate acoustic signal, and the modulated target signal. That is, the signal processing device 10 converts the non-target signal and the modulated target signal into an output acoustic signal. Specifically, the signal processing device 10 combines two or more acoustic signals from the non-target signal and the modulated target signal, or extracts or separates an acoustic signal from at least one of the non-target signal and the modulated target signal. The method of combining the acoustic signals is not limited, and may include, for example, signal summation, HRTF (Head Related Transfer Function) convolution, convolution with a transfer function that imparts position information of a sound source, or summation after performing these convolution processes. The signal processing device 10 may also perform at least one of amplification, volume adjustment, and D / A conversion of the output acoustic signal. On the other hand, if the asymmetric signal and the modulated target signal match the output format of the audio output device 30 (for example, if the asymmetric signal and the modulated target signal correspond to multi-channel audio signals associated with the speakers that make up a surround system as the audio output device 30), such conversion is not necessary. In this case, the asymmetric signal and the modulated target signal are treated as output audio signals.

[0070] In a second example of generating an output acoustic signal (S111), the input acoustic signal includes a plurality of acoustic signals having different characteristics. The signal processing device 10 selects one or more acoustic signals to which amplitude modulation is to be applied from the plurality of acoustic signals included in the input acoustic signal. The second example of generating an output acoustic signal (S111) can be understood by appropriately replacing the "intermediate acoustic signal" with the "input acoustic signal" in the description of the first example above.

[0071] In a third example of generating an output acoustic signal (S111), the signal processing device 10 performs amplitude modulation on the input acoustic signal. The amplitude modulation of the input acoustic signal is similar to the amplitude modulation of the target signal described in the first example of generating an output acoustic signal (S111). As a result, an amplitude change corresponding to the frequency of gamma waves is added to the input acoustic signal. Here, the signal processing device 10 may determine a modulation function used to amplitude modulate the input acoustic signal or a modulation depth for the amplitude modulation. The modulation function or modulation depth may be determined based on an input operation by a related person or an external instruction, or may be determined by an algorithm.

[0072] The signal processing device 10 generates an output acoustic signal based on a modulated input acoustic signal. That is, the signal processing device 10 converts the modulated input acoustic signal into an output acoustic signal. Specifically, when the modulated input acoustic signal is made up of multiple acoustic signals, the signal processing device 10 combines two or more of the multiple acoustic signals, or extracts or separates an acoustic signal from the modulated input acoustic signal. Details of the method for combining acoustic signals are as explained in the first example of generating an output acoustic signal (S111). On the other hand, when the modulated input acoustic signal matches the output format of the acoustic output device 30 (for example, when the modulated input acoustic signal corresponds to a multi-channel acoustic signal associated with each speaker that constitutes a surround system as the acoustic output device 30), such conversion is not necessary. In this case, the modulated input acoustic signal is treated as the output acoustic signal.

[0073] After step S111, the signal processing device 10 executes transmission of an output acoustic signal (S112). Specifically, the signal processing device 10 sends the output acoustic signal generated in step S111 to the acoustic output device 30. The acoustic output device 30 generates a sound corresponding to the output acoustic signal.

[0074] After step S112, the signal processing device 10 stores the listening history (S113). Specifically, the signal processing device 10 stores a listening history including at least one of the following in a listening history database (FIG. 5). Information about the feature quantity of the acoustic signal to which amplitude modulation was applied in step S111 Information about the modulation method of amplitude modulation performed in step S111 Information about the date and time when the sound output device 30 started to play the output sound signal sent in step S112 Information about the playback volume set in the audio output device 30 when the output audio signal sent in step S112 was played back. Information about the date and time when the sound output device 30 finished playing the output sound signal sent in step S112

[0075] After step S113, the signal processing device 10 ends the acoustic signal processing of FIG. The signal processing device 10 may perform the audio signal processing of Fig. 6 all at once on an input audio signal having a certain playback period (e.g., one piece of music content), or may repeatedly perform the audio signal processing of Fig. 6 for each predetermined playback section of the input audio signal (e.g., every 100 ms). Alternatively, the signal processing device 10 may continuously perform modulation processing on the input audio signal, such as modulation by analog signal processing, and output a modulated audio signal. The audio signal processing shown in Fig. 6 may be terminated in response to a specific termination condition (e.g., a certain amount of time has elapsed, an operation has been performed by a related person, or the output history of the modulated sound has reached a predetermined state).

[0076] (4-2) Information notification processing The information notification processing of this embodiment will be described. Fig. 7 is a flowchart of the information notification processing of this embodiment. Fig. 8 is a diagram showing an example of a score function used in the information notification processing of this embodiment. Fig. 9 is a diagram showing an example of presentation of target information in the information notification processing of this embodiment. Fig. 10 is a diagram showing an example of presentation of target information in the information notification processing of this embodiment. Fig. 11 is a diagram showing an example of presentation of target information in the information notification processing of this embodiment.

[0077] The information notification process in FIG. 7 starts when any of the following start conditions is met. The information notification process in Figure 7 was called by another process or an external instruction. The person involved performed the operation to call the information notification process shown in Figure 7. The signal processing device 10 is in a predetermined state (for example, power is turned on). The appointed date and time has arrived. A predetermined time has elapsed since a predetermined event (for example, the start-up of the signal processing device 10, the saving of a new listening history, or the previous execution of the information notification process in FIG. 7).

[0078] As shown in FIG. 7, the signal processing device 10 executes acquisition of a listening history (S120). Specifically, the signal processing device 10 acquires the patient's listening history from a listening history database (FIG. 5). As an example, the signal processing device 10 extracts from the listening history database a listening history in which at least one of the start date and time information or the end date and time information belongs to a period that is the target period for generating target information (hereinafter referred to as the "target period"). Here, the target period may be any of the following: The day, week, or month to which the information notification process in Figure 7 was executed A predetermined period going back from when the information notification process in Figure 7 was executed - for a period specified by the parties involved - Algorithmically specified period

[0079] After step S120, the signal processing device 10 generates target information (S121). Specifically, the signal processing device 10 generates target information based on the listening history acquired in step S120. The target information may include at least one of the following, for example. Listening history (e.g., the period during which the patient listened to sounds for cognitive improvement) - Information processed from listening history (e.g., statistical information) -Score of the degree of improvement in cognitive function according to the sounds the patient listened to

[0080] For example, in calculating the score, the signal processing device 10 can refer to a score function for deriving a score corresponding to an element included in the listening history (e.g., a listening period for the sound, a playback volume of the output audio signal, a modulation method for amplitude modulation applied to the audio signal, a feature amount of the audio signal to which amplitude modulation has been applied, or a combination thereof). The score function may be saved in the storage device 11 or may be stored in a storage device included in an external device (e.g., a server) accessible by the signal processing device 10.

[0081] Here, the score represents the degree of improvement in cognitive function and is defined to correlate with the estimated results of the amount of EEG induction caused by sound stimulation (i.e., the degree to which gamma waves were induced), an index calculated from the degree of EEG synchronization at each time (i.e., the duration for which gamma wave synchronization was achieved), or an index calculated from the EEG induction ratio at each time (i.e., the history of the degree to which gamma waves were induced compared to other EEG bands). As one example, the score is determined to increase (e.g., linearly increase) with an increase in the amount of EEG induction. As another example, the score is determined to increase little when the amount of EEG induction is below a first threshold, and to increase with an increase in the amount of EEG induction after the amount of EEG induction exceeds the first threshold. As yet another example, the score is determined to increase with an increase in the amount of EEG induction when the amount of EEG induction is below a second threshold, and to increase little (i.e., saturate) after the amount of EEG induction exceeds the second threshold. By measuring the brain wave elicitation level of a subject when the subject listens to various acoustic signals under various conditions using an electroencephalograph, it is possible to describe the relationship between the elements included in the listening history and the brain wave elicitation level.Based on the relationship between the elements included in the listening history and the brain wave elicitation level and the relationship between the score and the brain wave elicitation level, it is possible to create a score function for deriving a score corresponding to the elements included in the listening history.The score function may be defined as a mathematical formula with the elements of the listening history as arguments, or may be defined as a lookup table.

[0082] As an example, the signal processing device 10 calculates a score corresponding to the listening history using the score function shown in Fig. 8. In the score function of Fig. 8, the score is defined so that it increases or remains the same (in other words, it does not decrease) as the listening period increases. Also, in the score function of Fig. 8, the score is defined so that the score for a playback volume C1 is higher than the score for a playback volume C2 (lower than C1).

[0083] When the signal processing device 10 acquires a plurality of listening histories in step S120, the signal processing device 10 may calculate the transition of the cumulative score during the target period (i.e., the change in cumulative score over time) by cumulatively adding up the scores corresponding to each listening history.

[0084] When a target score for the patient during the target period is set, the signal processing device 10 may calculate the ratio of the cumulative score to the target score as the patient's achievement level. The target score corresponds to a standard score (i.e., a sound that the patient should hear) that the patient should achieve during the target period in order to improve cognitive function. The signal processing device 10 may calculate the achievement level at a single point in time, or may calculate the change in the achievement level over the target period (i.e., the change in the achievement level over time).

[0085] The signal processing device 10 may store the score calculation results in a database (not shown), thereby making it possible to obtain previously calculated scores in a short time and with reduced consumption of calculation resources.

[0086] After step S121, the signal processing device 10 executes notification of the target information (S122). Specifically, the signal processing device 10 transmits information necessary for presenting the target information generated in step S121 (for example, the target information itself, or information regarding the movement of light, images, sounds, characters, equipment, etc. that can be used to present the target information) to the related party terminal 70.

[0087] Information regarding the recipient of the target information (hereinafter referred to as "recipient information") is defined in association with information identifying the patient (e.g., patient ID). The recipient information is the account name, email address, or telephone number of the person concerned who is the subject of the notification. By using the recipient information, it is possible to identify the concerned person terminal 70 to which the information necessary for presenting the target information is to be sent.

[0088] The signal processing device 10 can notify the target information in at least one of the following formats, for example. Application notifications SNS (Social Networking Service) messages Email SMS (Short Service Message) messages Information that can be viewed after logging in to a specific application or SaaS (Software as a Service) (e.g., messages)

[0089] The related party terminal 70 presents the target information to the related party via an output device. The signal processing device 10 can express the target information using at least one of light, sound, text, images, and the movement of equipment or the like.

[0090] As described above, the target information may include a score. The score included in the target information may be at least one of the following: -Score for each listening history Cumulative score at any point during the period - Changes in cumulative scores during the target period - Achievement level at any point during the target period -Changes in achievement levels during the target period -Failure to reach the target score

[0091] As an example, as shown in FIGS. 9, 10 and 11, the related party terminal 70 can present target information to the related party by an image displayed on the display 81.

[0092] When the person involved has the authority to view the target information of multiple patients (typically when the person involved is a medical professional or a caregiver), the person involved terminal 70 displays screen P80 of Fig. 9 on the display 81. On the other hand, when the person involved only has the authority to view the target information of one patient (typically when the person involved is the patient himself / herself or a family member of the patient), the person involved terminal 70 displays screen P81 of Fig. 9 on the display 81.

[0093] The screen P80 displays a list of patients whose target information can be viewed. As shown in Fig. 9, the screen P80 includes an object A801.

[0094] The object A801 ​​receives a user instruction specifying which patient's target information is to be viewed. When the object A801 ​​is selected (or the name is clicked), the related party terminal 70 displays a screen P81 on the display 81.

[0095] The screen P81 is a menu screen for selecting which information to view from among the information about the patient. As shown in Fig. 9, the screen P81 includes objects A811 to A812. Note that the selectable objects or display contents on the menu screen may be limited depending on the authority of the person involved (for example, whether they are a doctor or not).

[0096] The object A811 receives a user instruction to display basic information about a patient. When the object A811 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the basic information about the patient.

[0097] The object A812 receives a user instruction to display the patient's score, etc. When the object A812 is selected, the related party terminal 70 displays a screen P82 shown in FIG.

[0098] Object A813 accepts a user instruction to display the results of the patient's cognitive function test. When object A811 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the results of the patient's cognitive function test.

[0099] The object A814 receives a user instruction to display the patient's medical record. When the object A814 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the results of the patient's medical record.

[0100] The screen P82 is a dashboard screen related to the patient's hearing status. As shown in Fig. 10, the screen P82 includes objects A821 to A827. Note that the selectable objects or display contents on the dashboard screen may be limited depending on the authority of the person involved (for example, whether they are a doctor or not).

[0101] Object A821 displays the patient's achievement level for today (that is, the viewing date of screen P82). Object A821 expresses the patient's achievement level using a pie chart and numbers.

[0102] Object A822 accepts a user instruction to display a detailed interview history of the patient. When object A822 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the detailed interview history of the patient.

[0103] Object A823 displays the patient's past achievement history in the form of a bar graph. The achievement history displayed in object A823 can be switched between yearly, monthly, or weekly units. When a bar representing the achievement level of any day in object A823 is selected, the related party terminal 70 displays screen P83 of FIG. 11 on the display 81.

[0104] The object A824 accepts a user instruction to display the history of doctor comments. When the object A824 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the history of doctor comments.

[0105] Object A825 accepts a user instruction to display the results of the patient's cognitive function test. When object A825 is selected, the related party terminal 70 displays a screen (not shown) on the display 81 that displays the results of the patient's cognitive function test.

[0106] Object A826 accepts a user instruction to display statistical information. When object A826 is selected, the related party terminal 70 displays a screen (not shown) displaying statistical information on the display 81. Here, the statistical information corresponds to the result of statistical processing of listening histories or scores collected for multiple patients. The listening histories or scores of multiple patients may be collected by an external device (for example, a server not shown) from the signal processing device 10 and other signal processing devices having similar functions to the signal processing device 10. The related party terminal 70 may display the statistical information in comparison with the listening histories or scores of the patients.

[0107] The object A827 accepts a user instruction to display an edit screen for a message to be sent. When the object A827 is selected, the related party terminal 70 displays an edit screen for a message to be sent (not shown) on the display 81.

[0108] The screen P83 is a screen that displays the patient interview status on a specific date selected by the person concerned. As shown in Fig. 11, the screen P83 includes objects A831 to A834.

[0109] Object A831 displays the patient's achievement level for a selected specific date. Object A831 represents the patient's achievement level using a pie chart and numbers.

[0110] Object A832 uses a line graph to represent the progress of the patient's achievement level on a selected date. Object A832 clearly indicates on the time axis the period during which the rate of increase in achievement level per unit time was high ("high efficiency"), the period during which the rate of increase in achievement level per unit time was low ("low efficiency"), the period during which processing of the input audio signal (amplitude modulation) was disabled ("processing OFF"), and the period during which playback of the output audio signal was stopped ("audio OFF"). Note that the efficiency classification may be divided into three or more stages instead of two. The current time may also be displayed.

[0111] Object A833 accepts a user instruction to shift the selected date forward by one day. When object A833 is selected, the related party terminal 70 updates the contents of objects A831 to A832.

[0112] Object A834 accepts a user instruction to shift the selected date forward by one day. When object A834 is selected, the related party terminal 70 updates the contents of objects A831 to A832.

[0113] 9 to 11 show examples in which target information relating to the listening situation of one selected patient is displayed. However, the present invention is not limited to this. The signal processing device 10 may transmit target information relating to multiple patients to the related party terminal 70, and the related party terminal 70 may simultaneously display the target information relating to the multiple patients. For example, the related party terminal 70 may display information on screen P82 relating to each of the multiple patients side by side on the display 81. Furthermore, the number of information items, display size, or amount of information displayed on the display 81 may be changed depending on the number of patients for which information is to be displayed. This makes it possible to efficiently display target information relating to multiple patients on a single screen.

[0114] The target information may be presented on an app screen or a website screen, or may be presented using a viewing screen for emails or other messages (e.g., SNS messages, SMS messages, etc.), or may be displayed on the screen of a smart speaker, television, smartwatch, car navigation system, etc.

[0115] Alternatively, the participant terminal 70 may express the score by sound output from a speaker (an example of an output device).

[0116] After step S122, the signal processing device 10 ends the information notification process of FIG.

[0117] (5) Summary As described above, the signal processing device 10 of this embodiment outputs an acoustic signal (i.e., an output acoustic signal) that is generated by amplitude modulation and has an amplitude change corresponding to the frequency of gamma waves, and stores information about the output acoustic signal (e.g., a listening history). The signal processing device 10 transmits a notification based on the stored information to a device external to the signal processing device 10 (i.e., the relevant person's terminal 70). This allows the relevant person to understand the patient's listening status for sounds intended to improve cognitive function.

[0118] The signal processing device 10 may transmit a notification to at least one of a device owned by the patient (i.e., the user of the sound output device 30), a device owned by the patient's family, a device owned by the patient's caregiver, or a device owned by a medical professional. This allows at least one of the patient himself / herself, the patient's family, the patient's caregiver, or the medical professional to understand the patient's hearing status.

[0119] The information about the acoustic signal stored by the signal processing device 10 may include information about at least one of the period during which the acoustic signal was output, the modulation method applied to the acoustic signal, the loudness (volume) of the sound emitted in response to the acoustic signal, and the feature quantity of the acoustic signal, thereby enabling relevant parties to be notified of details of the patient's hearing condition.

[0120] The notification may include information indicating at least one of the listening history and the score determined based on the listening history, thereby enabling the relevant person to grasp the details of the listening situation of the patient.

[0121] The output acoustic signal may have amplitude variations corresponding to frequencies between 35 Hz and 45 Hz, which may induce gamma waves in the patient's brain, resulting in improved cognitive function.

[0122] (6) Variations A modification of this embodiment will now be described.

[0123] (6-1) Variation 1 A description will be given of Modification 1. Modification 1 is an example in which notification of target information is performed conditionally.

[0124] (6-1-1) Information notification processing The following describes the information notification process of Modification 1. Fig. 12 is a flowchart of the information notification process of Modification 1.

[0125] The information notification process of the first modification starts when the same start condition as that of the information notification process of this embodiment is met.

[0126] 12, the signal processing device 10 acquires a listening history (S120) and generates target information (S121). Details of the acquisition of a listening history (S120) and the generation of target information (S121) are as described in the present embodiment.

[0127] After step S121, the signal processing device 10 executes a determination of the notification condition (S223). Specifically, the signal processing device 10 acquires information corresponding to the determination material and determines whether or not a predetermined notification condition is met based on the acquired information. The notification condition may be editable by the person concerned. The notification condition may be defined for each person concerned.

[0128] As a first example of determining whether a notification condition is met (S223), the signal processing device 10 may acquire a listening history collected about a patient and determine whether the acquired listening history satisfies a predetermined notification condition. For example, if no listening history has been recorded for even one day in the last week, or if a listening history has been recorded every day in the last three days, the signal processing device 10 may determine that the notification condition is met.

[0129] As a second example of determining whether the notification condition is met (S223), the signal processing device 10 may acquire a score calculated for the patient and determine whether the acquired score meets a predetermined notification condition. For example, if the average of the achievement level for each day in the most recent week is below or above a threshold, the signal processing device 10 may determine that the notification condition is met.

[0130] As a third example of determining the notification condition (S223), the signal processing device 10 may acquire the results of a cognitive function test for the patient and determine whether the acquired results satisfy a predetermined notification condition. For example, if the results of the cognitive function test indicate that the patient's cognitive function class has changed (improved or worsened), the signal processing device 10 may determine that the notification condition is met. Note that the signal processing device 10 may acquire the results of the cognitive function test from an external device, or may acquire the results by the signal processing device 10 itself administering the cognitive function test to the patient. In other words, the signal processing device 10 may be equipped with a function for conducting a cognitive function test (for example, a function for measuring cognitive function through voice dialogue with the patient).

[0131] If it is determined in step S223 that the notification condition is met, the signal processing device 10 executes notification of the target information (S122). Details of the notification of the target information (S122) are as described in the present embodiment.

[0132] After step S122, the signal processing device 10 ends the information notification process of FIG. Furthermore, if it is determined in step S223 that the notification condition is met, the signal processing device 10 ends the information notification process of FIG.

[0133] (6-1-2) Summary As described above, the signal processing device 10 of the first modification conditionally notifies the target information. For example, the signal processing device 10 may transmit a notification in response to the listening history or the score satisfying a predetermined condition. This provides timely feedback to relevant parties about the patient's signs of abandoning listening to the sounds or the patient's efforts, making it easier for the relevant parties to encourage the patient to continue listening to the sounds.

[0134] Alternatively, the signal processing device 10 may acquire the result of a cognitive function test of the patient and transmit a notification when the result of the cognitive function test satisfies a predetermined condition. This can prompt relevant parties to consider future actions for the patient (for example, setting a goal for listening to sounds to improve cognitive function) using the result of the cognitive function test as an opportunity.

[0135] (6-2) Variation 2 A description will be given of Modification 2. Modification 2 is an example in which target information is generated taking into consideration the patient's listening environment.

[0136] (6-2-1) Sound system configuration The configuration of the audio system will be described below. Fig. 13 is a block diagram showing the configuration of the audio system of the second modification.

[0137] As shown in FIG. 13, the sound system 2 includes a signal processing device 10, a sound output device 30, a sound source device 50, a related party terminal 70, and a sensor 90.

[0138] The signal processing device 10 and the sensor 90 may be connected to each other via a network (for example, the Internet or an intranet), or may be connected to each other via a predetermined interface (for example, a wireless interface using Bluetooth or the like).

[0139] The sensor 90 senses the listening environment of the patient. Specifically, the sensor 90 measures physical quantities related to at least one of the following: The patient's listening position (particularly, the relative position with respect to the sound output device 30) Sound pressure generated at the patient's listening position by the sound emitted from the sound output device 30 The patient's behavior when listening to the sound emitted from the sound output device 30 (for example, whether they are concentrating on listening or whether their attention is elsewhere)

[0140] The sensors 90 may include, for example, at least one of the following: - Devices that constitute radio wave positioning systems (such as Bluetooth AoA (Angle of Arrival)) (for example, devices that transmit or receive radio signals) Ranging sensors (e.g., ultrasonic sensors or LIDAR (Light Detection And Ranging)) Pressure sensors embedded in, for example, chairs, beds, or floors Camera (visible light camera or infrared camera) Microphone

[0141] (6-2-2) Information Processing The information processing of the second modification will be described.

[0142] (6-2-2-1) Acoustic signal processing A description will be given of the acoustic signal processing of Modification 2. Fig. 14 is a flowchart of the acoustic signal processing of Modification 2.

[0143] The acoustic signal processing of the second modification example starts when the same start condition as that of the acoustic signal processing of this embodiment is met.

[0144] 14, the signal processing device 10 acquires an input acoustic signal (S110), generates an output acoustic signal (S111), and sends out the output acoustic signal (S112). Details of the acquisition of the input acoustic signal (S110), the generation of the output acoustic signal (S111), and the sending out of the output acoustic signal (S112) are as described in the present embodiment.

[0145] After step S212, the signal processing device 10 determines the listening environment (S214). Specifically, the signal processing device 10 acquires the sensing result of the sensor 90. The signal processing device 10 determines the listening environment of the patient based on the sensing result. The listening environment of the patient may include at least one of the following, for example. The distance from the sound output device 30 to the patient's listening position, or the classification result thereof The direction from the sound output device 30 to the patient's listening position, or the classification result thereof The sound pressure generated at the patient's listening position by the sound emitted from the sound output device 30 (i.e., the degree to which the sound emitted from the sound output device 30 is attenuated), or the classification result thereof. The patient's behavior when listening to the sound emitted from the sound output device 30 (for example, the type of activity, such as sitting, lying down, exercising, or using a smartphone)

[0146] After step S214, the signal processing device 10 executes saving of the listening history (S113). Specifically, the signal processing device 10 stores a listening history including information related to the determination result in step S214 (hereinafter referred to as "listening environment information") in a listening history database. Other elements that can be included in the listening history are as described in the present embodiment.

[0147] After step S113, the signal processing device 10 ends the acoustic signal processing of FIG. The signal processing device 10 may perform the audio signal processing of Fig. 14 all at once on an input audio signal having a certain playback period (e.g., music content of one song), or may repeatedly perform the audio signal processing of Fig. 14 for each predetermined playback section of the input audio signal (e.g., every 100 ms). Alternatively, the signal processing device 10 may continuously perform modulation processing on the input audio signal, such as modulation by analog signal processing, and output a modulated audio signal. The audio signal processing shown in Fig. 14 may be terminated in response to a specific termination condition (e.g., a certain amount of time has elapsed, an operation has been performed by a related person, or the output history of the modulated sound has reached a predetermined state).

[0148] (6-2-2-2) Information notification processing The information notification process of the second modification will be described.

[0149] The information notification process of the second modification starts when the same start condition as that of the information notification process of the present embodiment is met.

[0150] The signal processing device 10 executes the acquisition of the listening history (S120). Details of the acquisition of the listening history (S120) are as described in the present embodiment.

[0151] After step S120, the signal processing device 10 generates target information (S121). Specifically, the signal processing device 10 generates target information based on the listening history acquired in step S120. For example, the generated target information may include listening environment information or information obtained by processing the listening environment information (e.g., statistical information). Furthermore, similar to the present embodiment, the signal processing device 10 may calculate a score corresponding to the listening history using a score function. In the second modification, the signal processing device 10 may further correct the score corresponding to the listening history according to the listening environment information included in the listening history.

[0152] As a first example of generating target information (S121), the signal processing device 10 corrects the score so that it decreases or remains the same as the distance from the sound output device 30 to the patient's listening position. As an example, assuming that the score before correction is the same, the signal processing device 10 corrects the score so that when the distance is a first value, the score is smaller than when the distance is a second value (the second value is smaller than the first value). At listening positions farther from the sound output device 30, there is a possibility that the sound pressure reaching the patient will decrease and the degree of modulation will be relatively reduced due to room reverberation or external sounds, which may reduce the effect of listening to improve cognitive function. However, by performing such a correction, a more reasonable score can be obtained.

[0153] As a second example of generating target information (S121), the signal processing device 10 corrects the score to decrease or maintain the same value in response to an increase in the difference between the direction from the sound output device 30 to the patient's listening position and the direction of the directivity of the sound output device 30 (hereinafter referred to as the "directional difference"). As an example, assuming that the score before correction is the same, the signal processing device 10 corrects the score so that when the directional difference is a first value, the score is smaller than when the directional difference is a second value (the second value is smaller than the first value). The amount of correction may be determined according to the directional characteristics (gain characteristics) of the sound output device 30. If the listening position is in a direction away from the directional direction of the sound output device 30, the sound pressure reaching the patient or the modulation level may decrease, which may reduce the effect of improving cognitive function through listening. However, by performing such correction, a more appropriate score can be obtained.

[0154] As a third example of generating target information (S121), the signal processing device 10 corrects the score to decrease or maintain the same score in response to a decrease in sound pressure or a decrease in modulation level caused at the patient's listening position by the sound emitted from the sound output device 30. As an example, assuming that the score before correction is the same, the signal processing device 10 corrects the score so that when the sound pressure is a first value, the score is smaller than when the sound pressure is a second value (the second value is greater than the first value). If a decrease in the sound pressure or modulation level reaching the patient occurs, there is a possibility that the effect of improving cognitive function through listening will be reduced, but by performing such a correction, a more reasonable score can be obtained.

[0155] As a fourth example of generating target information (S121), a correction amount is determined in advance for each patient's behavior (for example, listening situation, wakefulness state, or stress level) when listening to the sound emitted from the sound output device 30, and the signal processing device 10 corrects the score according to the correction amount. Although the effect of improving cognitive function due to listening may vary depending on the patient's behavior when listening to the sound emitted from the sound output device 30, performing such a correction makes it possible to obtain a more appropriate score.

[0156] After step S120, the signal processing device 10 executes notification of the target information (S122). Details of the notification of the target information (S122) are as described in the present embodiment.

[0157] (6-2-3) Summary As described above, the signal processing device 10 of the second modification may determine the listening environment of the patient for the sound corresponding to the output acoustic signal and generate target information based on the determination result. This makes it possible to notify relevant parties of information about the listening environment of the patient, or to notify relevant parties of a score calculated in consideration of the influence of the listening environment of the patient on the effect of improving cognitive function.

[0158] (7) Other variations The storage device 11 may be connected to the signal processing device 10 via a network. The display 21 may be integrated with the signal processing device 10. The storage device 71 may be connected to the participant terminal 70 via a network. The display 81 may be integrated with the participant terminal 70.

[0159] Each step of the above information processing can be executed by either the signal processing device 10 or the related party terminal 70. Some of the steps of the above information processing may be executed by the related party terminal 70 or an external device (e.g., a server) not shown. For example, the signal processing device 10 may store the listening history in the external device, the external device may generate target information based on the listening history, and the signal processing device 10 may acquire and notify the target information generated by the external device. In this case, the signal processing device 10 may omit the process of S120 in Fig. 7 and acquire the score from the external device in S121.

[0160] The above description has shown an example in which the signal processing device 10 is connected to one sound output device 30. The signal processing device 10 may be connected to a plurality of sound output devices 30, and in this case, the signal processing device 10 may be configured to be able to select to which sound output device 30 an output sound signal is to be sent.

[0161] The above description has shown an example in which the signal processing device 10 is connected to one sound source device 50. The signal processing device 10 may be connected to a plurality of sound source devices 50, and in this case, the signal processing device 10 may be configured to be able to select from which sound source device 50 the input acoustic signal is to be acquired.

[0162] The above description has focused on the case where one signal processing device 10 is associated with one patient. However, this is not limiting, and one signal processing device 10 may be associated with multiple patients. For example, consider a case where multiple patients (users listening to the sound output from the sound output device 30) are present in a room where the signal processing device 10 and a speaker serving as the sound output device 30 are installed. In this case, the signal processing device 10 identifies the multiple patients present in the room and updates the listening history database corresponding to each patient based on the output result of the sound signal from the signal processing device 10. This makes it possible to easily grasp the listening status of each patient even when multiple patients listen to the sound based on the sound signal output from one signal processing device 10. As a method for the signal processing device 10 to identify patients listening to the sound emitted from the sound output device 30 (e.g., patients in the same room as the sound output device 30), for example, the signal processing device 10 may acquire registered ID information when the patient enters the room. Another method is to identify patients present near the sound output device 30 by performing wireless communication, such as Bluetooth, between a terminal device held by the patient and the sound output device 30. However, the present invention is not limited to these methods, and the signal processing device 10 can use various existing identification methods.

[0163] Furthermore, multiple signal processing devices 10 may be associated with one patient. For example, consider a case where a signal processing device 10 and a speaker serving as an audio output device 30 are installed in each of multiple rooms in a facility where patients reside. In this case, the multiple signal processing devices 10 in the facility access a common listening history database. Each signal processing device 10 identifies a patient in the same room as the audio output device 30 connected to that signal processing device 10, and updates the listening history database corresponding to that patient based on the output result of the audio signal from that signal processing device 10. This makes it possible to easily grasp the listening status of a single patient even when the patient listens to sounds based on audio signals output from multiple signal processing devices 10. The signal processing device 10 can use the various methods described above to identify patients.

[0164] Similarly, a plurality of signal processing devices 10 may be associated with a plurality of patients. That is, each of the plurality of signal processing devices 10 may update a listening history database for each of the plurality of patients who listen to sounds based on acoustic signals output from the signal processing device 10. Then, the signal processing device 10 may transmit a notification based on information acquired from the listening history database to the related party terminal 70 associated with each patient.

[0165] The signal processing device 10 may be equipped with a function for conducting a cognitive function test (for example, a function for measuring cognitive function through voice dialogue with a patient). In this case, the signal processing device 10 may notify the relevant party terminal 70 of target information including information indicating at least one of the patient's answers to the cognitive function test and the results of the cognitive function test. This allows the relevant party to timely grasp the patient's cognitive function in addition to the patient's hearing status.

[0166] In the second modification, an example is shown in which the score is calculated taking into consideration listening environment information. The signal processing device 10 may calculate the score taking into consideration other information. As a first example, the signal processing device 10 may acquire information about the attributes of a patient listening to a sound corresponding to the output acoustic signal (hereinafter referred to as "attribute information") and calculate a score based on the attribute information. The attribute information may include at least one of the patient's age, gender, hearing ability (which may include hearing ability in each frequency band or the ability to hear speech), dementia risk, past listening history (which may include the results of an analysis of the past listening history), preferences or intentions (commitment) for listening to sound, and the results of past electroencephalogram (EEG) measurements or psychological experiments, or feature quantities based on those results. This allows for a score that takes into account the influence of the patient's attributes on the improvement effect of cognitive function. The signal processing device 10 may use a sensor 90 (particularly, a device constituting a radio wave positioning system or a camera) to identify the patient. A hearing test function may be implemented in the signal processing device 10 or the related party terminal 70 to measure the patient's hearing ability. As a second example, the signal processing device 10 may acquire the results of a cognitive function test of a patient who listens to a sound corresponding to the output acoustic signal (i.e., the patient's current cognitive function), and calculate a score based on the test results. This makes it possible to obtain a score that takes into account the influence of the patient's current cognitive function on the improvement effect of cognitive function. The signal processing device 10 or the related party terminal 70 may be equipped with a function for conducting a cognitive function test (e.g., a function for measuring cognitive function through voice dialogue with the patient).

[0167] The signal processing device 10 may change the method of generating an output audio signal (e.g., modulation method, synthesis ratio of two or more audio signals, method of determining an audio signal to which amplitude modulation is applied, etc.) or the playback volume of a sound corresponding to the output audio signal, depending on the patient's score. As a first example, the signal processing device 10 may generate an output audio signal corresponding to a sound that will result in a higher score being attained by listening, or may increase the playback volume, as the remaining time until the expiration of a period for which a target score is set is shorter. As a second example, the signal processing device 10 may generate an output audio signal corresponding to a sound that will result in a higher score being attained by listening, or may increase the playback volume, as the patient's current level of achievement is lower. Furthermore, when the level of achievement of the score does not meet a predetermined standard, the signal processing device 10 may automatically play an amplitude-modulated audio signal or may play a predetermined audio signal that has a high brainwave elicitation effect (e.g., a beep or chirp sound unrelated to the input audio signal and having a frequency corresponding to gamma waves). This makes it possible to assist the patient in achieving the target score without paying attention to the score achievement status.

[0168] In the above description, an example has been given in which the score is calculated based on the listening history stored by the signal processing device 10. However, in addition to or instead of the listening history, the score may be calculated based on an audio signal received by a microphone (e.g., the intensity, duration, waveform, or a combination thereof of the audio signal). Such a microphone may be installed closer to the patient's listening position than the sound output device 30, and may be, for example, a microphone included in a terminal used by the patient, or a microphone that can be worn on the patient's body (e.g., near the ear).

[0169] An algorithm for determining a method for generating an output audio signal (e.g., a modulation method, a combining ratio of two or more audio signals, a method for determining an audio signal to which amplitude modulation is applied, etc.), or a default method for generating an output audio signal, may be changed by updating the firmware of the signal processing device 10. The firmware is updated, for example, by the signal processing device 10 communicating with an external device (e.g., a server not shown). Similarly, a method for generating target information (which may include a method for calculating a score), a method for notifying the target information, a method for presenting the target information, etc. may also be changed by updating the firmware of the signal processing device 10. The contents of the firmware update of the signal processing device 10 may be notified to relevant parties in advance, and the signal processing device 10 may provide an operating mode that allows relevant parties to compare sounds before and after the update.

[0170] Although an example has been given in which the input sound signal is acquired from the sound source device 50, the input sound signal may be generated by receiving environmental sound with a microphone. The input sound signal may be read from the storage device 11 or a removable medium connected to the signal processing device 10. The sound source device 50 (including the sound source device 50 built into the signal processing device 10) may generate the input sound signal each time.

[0171] In the above description, an example has been given in which an acoustic signal derived from an input acoustic signal is modulated. However, an output acoustic signal may also be generated based on the input acoustic signal and a modulated acoustic signal that is not derived from the input acoustic signal.

[0172] The above description has focused on an example in which the modulation function has a frequency of 35 Hz or more and 45 Hz or less. However, the modulation function used by the signal processing device 10 is not limited to this, and any modulation function that affects the induction of gamma waves in the listener's brain may be used. For example, the modulation function may have a frequency of 25 Hz or more and 140 Hz or less. Furthermore, for example, the frequency of the modulation function may change over time, and the modulation function may partially have a frequency below 35 Hz or a frequency higher than 45 Hz.

[0173] In the above description, the output acoustic signal generated by the signal processing device 10 is output to the acoustic output device 30, which emits sound and allows a patient to hear it. However, the output destination of the output acoustic signal from the signal processing device 10 is not limited to this. For example, the signal processing device 10 may output the output acoustic signal to an external storage device or information processing device via a communication network or by broadcasting. In this case, the signal processing device 10 may output an unmodulated input acoustic signal to the external device along with the output acoustic signal generated by modulation. This allows the external device to arbitrarily select and play either the unmodulated acoustic signal or the modulated acoustic signal. Furthermore, the signal processing device 10 may output information indicating the content of the modulation process to an external device together with the output acoustic signal. The information indicating the content of the modulation process includes, for example, any of the following: - Information indicating the sound source corresponding to the modulated acoustic signal - Information indicating the channel corresponding to the modulated audio signal - Information that indicates the characteristics of the modulated acoustic signal Information indicating modulation function -Information indicating modulation depth Volume information This allows the external device to change the method of reproducing the audio signal depending on the content of the modulation process.

[0174] In the above description, an example has been shown in which the signal processing device 10 performs modulation processing including amplitude modulation to generate a modulated acoustic signal. However, the signal processing device 10 may also acquire a modulated acoustic signal generated by an external device and transmit an output acoustic signal based on the modulated acoustic signal. In this case, the signal processing device 10 may acquire information necessary for generating a listening history (e.g., modulation method information or feature information) from the external device.

[0175] The related party terminal 70 may collect other information (for example, TV viewing status (type of program, viewing time, etc.), vital data of the patient obtained from a wearable terminal) and present the collected information together with the target information.

[0176] Furthermore, when the signal processing device 10 acquires additional information (for example, an ID3 tag in an MP3 file) along with the input audio signal, the signal processing device 10 may modify the additional information and output it to an external device together with the output audio signal.

[0177] Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited to the above-described embodiments. Furthermore, the above-described embodiments can be improved or modified in various ways without departing from the spirit of the present invention. Furthermore, the above-described embodiments and modifications can be combined. [Explanation of symbols]

[0178] 1: Sound system 2: Sound system 10: Signal processing device 11:Storage device 12: Processor 13: Input / output interface 14: Communication interface 21: Display 30: Sound output device 50: Sound source device 70: Stakeholder terminal 71 :Storage device 72: Processor 73: Input / output interface 74: Communication interface 81: Display 90: Sensor

Claims

1. 1. A signal processing device, comprising: an output means for outputting an acoustic signal generated by amplitude modulation, the acoustic signal having an amplitude change corresponding to the frequency of gamma waves; a storage means for storing information on the acoustic signal output by the output means; a transmitting means for transmitting a notification based on the information stored in the storage means to a device external to the signal processing device; A signal processing device comprising:

2. 2. The signal processing device according to claim 1, wherein the transmitting means transmits the notification to at least one of a device owned by a user who listens to the acoustic signal output by the output means, a device owned by a family member of the user, a device owned by a caregiver of the user, and a device owned by a medical professional.

3. 2. The signal processing device according to claim 1, wherein the information about the acoustic signal stored by the storage means includes information indicating at least one of a period during which the acoustic signal was output by the output means, a modulation method for the output acoustic signal, a magnitude of the output acoustic signal, and a feature amount of the output acoustic signal.

4. The signal processing device according to claim 1 , wherein the notification transmitted by the transmitting means includes information indicating at least one of an output history of the acoustic signal by the output means and a score determined based on the output history of the acoustic signal.

5. 2. The signal processing device according to claim 1, wherein the transmission means transmits the notification in response to at least one of an output history of the acoustic signal by the output means and a score determined based on the output history of the acoustic signal satisfying a predetermined condition.

6. having an acquisition means for acquiring the results of a cognitive function test; The signal processing device according to claim 1 , wherein the transmitting means transmits the notification in response to the result of the cognitive function test acquired by the acquiring means satisfying a predetermined condition.

7. The signal processing device according to claim 1 , wherein the transmitting means transmits, together with the notification, information indicating at least one of the answers to a cognitive function test and the results of the cognitive function test.

8. 2. The signal processing device according to claim 1, wherein the acoustic signal output by said output means has a change in amplitude corresponding to a frequency between 35 Hz and 45 Hz.

9. A signal processing device according to any one of claims 1 to 8; means for making a user hear a sound corresponding to the acoustic signal output by the signal processing device; A cognitive function improvement system comprising:

10. A signal processing method performed by a signal processing device, comprising: outputting an acoustic signal generated by amplitude modulation, the acoustic signal having amplitude variations corresponding to the frequency of gamma waves; storing information about the output acoustic signal; sending a notification based on the stored information to a device external to the signal processing device; Signal processing methods.

11. A program for causing a computer to function as each of the means of the signal processing device according to any one of claims 1 to 8.

Images