Signal processing device, cognitive function improvement system, signal processing method, and program

Abstract

A signal processing device according to one embodiment of the present disclosure is provided with: a receiving means for receiving an input audio signal; a generation unit that generates a modulated audio signal having a change in amplitude corresponding to the frequency of the gamma wave by performing modulation processing including amplitude modulation on the input audio signal received by the reception unit; and an output means for outputting content information including the modulated audio signal generated by the generation means in association with modulation information relating to a modulation process performed in order to generate the modulated audio signal.

Description

Technical Field

[0001] The present disclosure relates to a signal processing device, a cognitive function improvement system, a signal processing method, and a program. Background Art

[0002] There is a research report that when a living being is made to perceive pulsed sound stimulation at a frequency of about 40 times per second and gamma waves are induced in the brain of the living being, it is effective in improving the cognitive function of the living being (see Non-Patent Document 1).

[0003] Gamma waves are neural oscillations whose frequencies are included in the gamma frequency band (25 Hz to 140 Hz) among neural oscillations obtained by capturing periodic neural activities of the cerebral cortex using electrophysiological methods such as electroencephalography and magnetoencephalography.

[0004] Patent Document 1 discloses that the volume is adjusted by increasing or decreasing the amplitude of a sound wave or a sound channel to generate a rhythmic stimulation corresponding to a stimulation frequency for inducing tuning of brain waves.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application No. 2020-501853

[0008] Non-patent literature

[0009] Non-patent literature 1: Multi-sensory Gamma Stimulation Ameliorates Alzheimer's-Associated Pathology and Improves Cognition Cell 2019Apr4;177(2):256-271.e22.doi:10.1016 / j.cell.2019.02.014. Summary of the invention

[0010] Problem that the invention aims to solve

[0011] When playing content information including an audio signal, if it is unclear whether the content information includes an amplitude modulated audio signal and what kind of amplitude modulation is used when the content information includes an amplitude modulated audio signal, it is considered to cause inconvenience to the user. For example, a user who expects an improvement in cognitive function may not be able to easily determine what content information to listen to or watch for how long.

[0012] An object of the present disclosure is to provide a technology for improving user convenience in determining the content information to be broadcasted or determining the listening or viewing method of the content information.

[0013] Solutions for solving problems

[0014] A signal processing device according to one embodiment of the present invention comprises: a receiving unit that receives an input audio signal; a generating unit that generates a modulated audio signal having an amplitude change corresponding to the frequency of a gamma wave by performing a modulation process including amplitude modulation on the input audio signal received by the receiving unit; and an output unit that outputs content information of the modulated audio signal generated by the generating unit in correspondence with modulation information related to the modulation process performed to generate the modulated audio signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 : is a block diagram showing the structure of the cognitive function improving system according to the present embodiment.

[0016] Figure 2 This is a block diagram showing the structure of the content information playback device according to the present embodiment.

[0017] Figure 3 This is a block diagram showing the structure of the content information distribution device according to the present embodiment.

[0018] Figure 4 This is an explanatory diagram of one mode of this embodiment.

[0019] Figure 5 This is a flowchart of the content information distribution process according to the present embodiment.

[0020] Figure 6 : is a diagram showing an example of a score function used in the content information distribution process of the present embodiment.

[0021] Figure 7 This is a flowchart of the content information playback process of this embodiment.

[0022] Figure 8 This is a diagram showing an example of a screen displayed during the content information playback process according to the present embodiment.

[0023] Fig. 9 This is a diagram showing an example of a screen displayed during the content information playback process according to the present embodiment.

[0024] Fig.10 This is a flowchart of the content information distribution process of Modification Example 1.

[0025] Fig.11 This is a flowchart of the content information playback process of Modification Example 1.

[0026] Fig.12 This is a flowchart of the content information playback process of Modification Example 2. DETAILED DESCRIPTION

[0027] Hereinafter, one embodiment of the present invention will be described in detail based on the drawings. In the drawings for describing the embodiments, the same components are denoted by the same reference numerals in principle, and their repeated descriptions are omitted.

[0028] (1) Structure of cognitive function improvement system

[0029] The structure of the cognitive function improvement system is described. Figure 1 : is a block diagram showing the structure of the cognitive function improving system according to the present embodiment.

[0030] like Figure 1 As shown, the cognitive function improvement system 1 includes a content information playback device 10 and a content information distribution device 30 .

[0031] A transmission path for distributing content information is provided between the content information playback device 10 and the content information distribution device 30. The content information may be either broadcast content information or content information distributed on demand. In the present embodiment, the content information distributed includes sound content information. In addition, the content information distribution device 30 may also distribute content information including sound content information and moving image content information (e.g., TV programs, uploaded MP4 files). The transmission path may be either a transmission path for information communication or a transmission path for broadcasting. The transmission path for information communication may also include a network (e.g., the Internet or an intranet) or a specified interface (e.g., a wireless interface using Bluetooth, etc.). The transmission path for broadcasting may include, for example, a broadcast network for TV broadcasting.

[0032] The content information playback device 10 acquires content information distributed from the content information distribution device 30 and plays the content information. The content information playback device 10 is an example of a signal processing device. The content information playback device 10 is, for example, a TV, a set-top box, a radio, a music player, a smart phone, a PC, an electronic musical instrument, a telephone, a game console, or an amusement machine.

[0033] The content information distribution device 30 distributes content information to the content information playback device 10. The content information distribution device 30 is an example of a signal processing device.

[0034] (1-1) Structure of content information playback device

[0035] The structure of the content information playback device will be described. Figure 2This is a block diagram showing the structure of the content information playback device according to the present embodiment.

[0036] like Figure 2 As shown, the content information playback device 10 includes a storage device 11, a processor 12, an input / output interface 13, and a communication interface 14. The content information playback device 10 is connected to a display 21 and an audio output device 22.

[0037] The storage device 11 is configured to store programs and data. The storage device 11 is, for example, a combination of a ROM (Read Only Memory), a RAM (Random Access Memory), and a storage device (such as a flash memory or a hard disk). The programs and data may be provided via a network or recorded in a computer-readable recording medium.

[0038] The program includes, for example, the following programs.

[0039] ·OS (operating system) programs

[0040] · Programs that execute information processing applications

[0041] The data includes, for example, the following data.

[0042] · Databases referenced in information processing

[0043] Data obtained by executing information processing (i.e., the results of executing information processing)

[0044] The processor 12 is a computer that implements the functions of the content information playback device 10 by reading and executing the program stored in the storage device 11. In addition, at least part of the functions of the content information playback device 10 may also be implemented by one or more dedicated circuits. The processor 12 is, for example, at least one of the following.

[0045] CPU (Central Processing Unit)

[0046] GPU (Graphic Processing Unit)

[0047] ASIC (Application Specific Integrated Circuit)

[0048] FPGA (Field Programmable Array)

[0049] DSP (Digital Signal Processor)

[0050] The input / output interface 13 is configured to obtain information (eg, user instructions) from an input device connected to the content information playback apparatus 10 and output information (eg, image signals, audio signals, or control signals) to an output device connected to the content information playback apparatus 10 .

[0051] The input device is, for example, a physical button, a keyboard, a pointing device, a touch panel, or a combination thereof.

[0052] The output device is, for example, a display 21, an audio output device 22, or a combination thereof.

[0053] Furthermore, the input / output interface 13 can include hardware for signal processing, such as an A / D converter, a D / A converter, an amplifier, a mixer, and a filter.

[0054] The communication interface 14 is configured to control communication between the content information playback device 10 and an external device (eg, the content information distribution device 30 ).

[0055] The display 21 is configured to display an image (still image or moving image). The display 21 is, for example, a liquid crystal display or an organic EL display.

[0056] The audio output device 22 generates sound corresponding to the output audio signal and is, for example, a loudspeaker (which can include a speaker with a built-in amplifier (a driver speaker)), headphones, earphones, or a smart speaker.

[0057] The audio output device 22 is connected to the processor 12 via a specified interface capable of transmitting audio signals, and obtains an output audio signal from the processor 12. The interface is, for example, SPDIF (Sony Philips Digital Interface), HDMI (registered trademark) (High-Definition Multimedia Interface), a pin connector (RCA pin), or an audio interface for headphones. The interface may also be a wireless interface using Bluetooth (registered trademark) or the like. The audio signal in this embodiment includes either or both of an analog signal and a digital signal.

[0058] (1-2) Configuration of content information distribution device

[0059] The structure of the content information distribution device will be described. Figure 3 This is a block diagram showing the structure of the content information distribution device according to the present embodiment.

[0060] like Figure 3 As shown, the content information distribution device 30 includes a storage device 31, a processor 32, an input / output interface 33, and a communication interface 34. The content information distribution device 30 is connected to a sound source device 41.

[0061] The storage device 31 is configured to store programs and data. The storage device 31 is, for example, a combination of ROM, RAM, and a storage device (such as a flash memory or a hard disk). The programs and data may be provided via a network or recorded in a computer-readable recording medium.

[0062] The program includes, for example, the following programs.

[0063] ·OS program

[0064] · Programs that execute information processing applications

[0065] The data includes, for example, the following data.

[0066] · Databases referenced in information processing

[0067] ·The results of information processing

[0068] The processor 32 is a computer that implements the functions of the content information distribution device 30 by reading and executing the program stored in the storage device 31. In addition, at least part of the functions of the content information distribution device 30 may be implemented by one or more dedicated circuits. The processor 32 is, for example, at least one of the following.

[0069] CPU

[0070] GPU

[0071] ASIC

[0072] FPGA

[0073] DSP

[0074] The input / output interface 33 is configured to acquire information (eg, user instructions or audio signals) from an input device connected to the content information distribution apparatus 30 , and output information (eg, image signals) to an output device connected to the content information distribution apparatus 30 .

[0075] The input device is, for example, the sound source device 41, a physical button, a keyboard, a pointing device, a touch panel, or a combination thereof.

[0076] Output devices are, for example, displays, speakers, or a combination thereof.

[0077] The communication interface 34 is configured to control communication between the content information distribution device 30 and an external device (eg, the content information playback device 10 ).

[0078] The sound source device 41 is configured to generate an input audio signal and output the input audio signal.

[0079] (2) One mode of implementation

[0080] One mode of this embodiment will be described. Figure 4 This is an explanatory diagram of one mode of this embodiment.

[0081] like Figure 4 As shown, the content information distribution device 30 performs amplitude modulation processing on the audio signal. In addition, the content information distribution device 30 generates meta-information related to the content information for improving cognitive function. The meta-information can include, for example, information related to the amplitude modulation performed on the audio signal (hereinafter referred to as "modulation information"). The content information distribution device 30 distributes the content information including the audio signal (output audio signal) generated by the amplitude modulation processing and the meta-information to the content information playback device 10.

[0082] The content information playback device 10 plays the content information acquired from the content information distribution device 30. The content information playback device 10 causes the audio output device 22 to output the audio signal (output audio signal) included in the content information. The output audio signal has a change in amplitude corresponding to the frequency of the gamma wave, so when the user US10 listens to the sound based on the output audio signal, the gamma wave is induced in the brain of the user US10 (the brain wave is synchronized with the gamma frequency). As a result, the effect of improving cognitive function (for example, the treatment or prevention of cognitive impairment) can be expected. Here, the content information playback device 10 performs control related to the playback of the content information based on the meta-information acquired from the content information distribution device 30. As an example, the content information playback device 10 displays information based on the meta-information (for example, information related to the magnitude of the improvement effect of cognitive function that can be expected by viewing and listening to the content information) on the display 21. In addition, in the following description, "viewing and listening" is used to include the meaning of "listening". By confirming such information, the user US10 can easily determine whether to continue viewing the content information being played or to switch to other content information, or for what period of time the content information being played should be continued.

[0083] (3) Information processing

[0084] The information processing in this embodiment is described.

[0085] (3-1) Content Information Distribution Processing

[0086] The content information distribution process according to the present embodiment will be described. Figure 5 This is a flowchart of the content information distribution process according to the present embodiment. Figure 6 : is a diagram showing an example of a score function used in the content information distribution process of the present embodiment.

[0087] Figure 5 The content information distribution process is implemented by the processor 32 of the content information distribution device 30 reading out and executing the program stored in the storage device 31 .

[0088] Starts in response to any of the following start conditions being met Figure 5 Content information distribution processing.

[0089] ·Called out due to other processing or external instructions Figure 5 Content information distribution processing.

[0090] · The user of the content information playback device 10 performs a Figure 5 Operations of content information distribution processing.

[0091] The content information distribution device 30 is in a predetermined state (for example, the power is turned on).

[0092] ·The specified date and time has arrived.

[0093] · From a predetermined event (for example, activation of the content information distribution device 30, or Figure 5 A specified time has passed since the last execution of the content information distribution processing.

[0094] like Figure 5 As shown, the content information distribution device 30 performs acquisition of an input audio signal (S130).

[0095] Specifically, the content information distribution device 30 receives an input audio signal sent from the sound source device 41. Alternatively, the processor 32 may read content information including the input audio signal from the storage device 31 and extract (which may include demultiplexing and decoding) the input audio signal.

[0096] In step S130 , the content information distribution device 30 may further perform A / D conversion on the input audio signal.

[0097] The input audio signal corresponds to, for example, at least one of the following.

[0098] Music content information (e.g., singing, playing, or a combination thereof (i.e., music). It can include music content information accompanying the moving image content information.)

[0099] Voice content information (e.g., voice such as reading, commentary, broadcasting, radio drama, personal performance, conversation, monologue, or a combination thereof. Voice content information accompanying moving image content information may be included.)

[0100] Other audio content information (e.g., electronic, environmental, or mechanical sounds)

[0101] However, the singing or voice content information is not limited to the voice produced by the human vocal organs, and can include the voice generated by the voice synthesis technology.

[0102] After step S130, the content information distribution device 30 performs modulation of the audio signal (S131).

[0103] Specifically, the content information distribution device 30 generates an output audio signal by at least partially performing amplitude modulation on the input audio signal acquired in step S130 .

[0104] As a first example of modulation of an audio signal (S131), the content information distribution device 30 generates an intermediate audio signal including a plurality of audio signals having different characteristics based on an input audio signal. Here, the characteristics may be determined based on an input operation of a producer or provider of content information corresponding to the input audio signal, or an instruction from the outside, or may be determined by an algorithm. For example, the content information distribution device 30 may also determine the characteristics used to generate the intermediate audio signal based on the result of analyzing the input audio signal.

[0105] The feature may be, for example, at least one of the following.

[0106] · Characteristics of sound (especially quality)

[0107] Frequency characteristics

[0108] Characteristics of time

[0109] Amplitude characteristics

[0110] Output characteristics

[0111] The content information distribution device 30 selects one or more audio signals (hereinafter referred to as "object signals") to which amplitude modulation is to be applied from the multiple audio signals included in the intermediate audio signal. As to which audio signal is selected as the object signal, it can be determined based on the input operation of the producer or provider of the content information corresponding to the input audio signal, or an instruction from the outside, or it can be determined by an algorithm. For example, the content information distribution device 30 can also determine the object signal based on the characteristics of the audio signal (balance between voice and music, volume change, type of music, timbre or other characteristics). Thus, the content information distribution device 30 can select the object signal in a manner that makes the improvement effect of cognitive function obtained by modulation higher, or select the object signal in a manner that makes the sense of disharmony brought to the viewer of the content information (that is, the user of the content information playback device 10) smaller. However, the content information distribution device 30 can also process all of the multiple audio signals included in the intermediate audio signal as object signals.

[0112] The content information distribution device 30 performs amplitude modulation on the selected object signal. As an example, the content information distribution device 30 uses a modulation function having a frequency corresponding to a gamma wave (for example, a frequency of 35 Hz or more and 45 Hz or less) to perform amplitude modulation on the object signal. Specifically, when a modulation function having a periodicity of 35 Hz or more and 45 Hz or less is set to A(t), a function representing the waveform of the audio signal before modulation is set to X(t), and a function representing the waveform of the modulated audio signal is set to Y(t), it becomes: Y(t) = A(t)·X(t). Thus, a change in amplitude corresponding to the above frequency is added to the object signal. Here, the content information distribution device 30 may also determine the modulation function used in the amplitude modulation of the object signal or the modulation degree of the amplitude modulation. The modulation function or the modulation degree may be determined based on an input operation of a producer or provider of content information corresponding to the input audio signal, or an instruction from the outside, or may be determined by an algorithm. In addition, the content information distribution device 30 may determine a common modulation function or modulation degree for a plurality of target signals, or may determine a modulation function or modulation degree for each of a plurality of target signals individually.

[0113] The content information distribution device 30 generates an output audio signal based on an audio signal (hereinafter referred to as a "non-object signal") that is not selected as an object signal among a plurality of audio signals included in the intermediate audio signal and a modulated object signal. That is, the content information distribution device 30 converts the non-object signal and the modulated object signal into an output audio signal. Specifically, the content information distribution device 30 synthesizes two or more audio signals among the non-object signal and the modulated object signal, or extracts or separates the audio signal from at least one of the non-object signal and the modulated object signal. There is no limitation on the synthesis method of the audio signal, and it can include, for example, signal summing processing, HRTF (Head Related Transfer Function) convolution processing, convolution processing of a transfer function that assigns position information of a sound source, or summing processing after performing these convolution processing. In addition, the content information distribution device 30 can also perform at least one of amplification, volume adjustment, or D / A conversion of the output audio signal. On the other hand, when the non-object signal and the modulated object signal are consistent with the format of the distributable audio signal or the output format of the content information playback device 10 (for example, when the non-object signal and the modulated object signal are equivalent to the multi-channel audio signal associated with each speaker constituting the surround sound system as the content information playback device 10), such conversion is not required. In this case, the non-object signal and the modulated object signal are processed as the output audio signal.

[0114] In the second example of the modulation of the audio signal (S131), the input audio signal includes a plurality of audio signals having different characteristics. The content information distribution device 30 selects one or more audio signals to which amplitude modulation is applied from the plurality of audio signals included in the input audio signal. The second example of the modulation of the audio signal (S131) ​​can be understood by appropriately replacing the "intermediate audio signal" in the description of the first example with the "input audio signal".

[0115] In the third example of modulation of an audio signal (S131), the content information distribution device 30 performs amplitude modulation on the input audio signal. The amplitude modulation on the input audio signal is the same as the amplitude modulation on the object signal described in the first example of modulation of an audio signal (S131). Thus, a change in amplitude corresponding to the frequency of the gamma wave is added to the input audio signal. Here, the content information distribution device 30 may also determine a modulation function used in the amplitude modulation of the input audio signal or a modulation degree of the amplitude modulation. The modulation function or the modulation degree may be determined based on an input operation of a producer or provider of content information corresponding to the input audio signal, or an instruction from the outside, or may be determined by an algorithm.

[0116] The content information distribution device 30 generates an output audio signal based on the modulated input audio signal. That is, the content information distribution device 30 converts the modulated input audio signal into an output audio signal. Specifically, in the case where the modulated input audio signal is composed of multiple audio signals, the content information distribution device 30 synthesizes two or more audio signals among the multiple audio signals, or extracts or separates the audio signal from the modulated input audio signal. The details of the audio signal synthesis method are as described in the first example of audio signal modulation (S131). On the other hand, in the case where the modulated input audio signal is consistent with the form of the distributable audio signal or the output form of the content information playback device 10 (for example, in the case where the modulated input audio signal is equivalent to a multi-channel audio signal associated with each speaker constituting the surround sound system of the content information playback device 10), such conversion is not required. In this case, the modulated input audio signal is processed as an output audio signal.

[0117] After step S131, the content information distribution device 30 performs generation of meta information (S132).

[0118] Specifically, the content information distribution device 30 generates meta-information corresponding to the process performed in the modulation (S131) ​​of the audio signal. The meta-information can include, for example, at least one of the following information.

[0119] Information (e.g., a flag) indicating that playback of a sound based on an output audio signal (that is, an audio signal to which amplitude modulation is applied) can be selected.

[0120] Modulation information

[0121] Feature quantity information

[0122] Ideal viewing and listening conditions for the content information (e.g. playback volume)

[0123] Category of content information

[0124] ·Fraction

[0125] The modulation information is information related to the processing performed to generate the output audio signal included in the content information. For example, the modulation information can include information related to at least one of the following.

[0126] Modulation function used in amplitude modulation

[0127] Modulation

[0128] Audio signal to which amplitude modulation is applied

[0129] Audio signal without amplitude modulation applied

[0130] Here, when an intermediate audio signal including a plurality of audio signals is generated based on an input audio signal and it is determined whether or not amplitude modulation is applied to each of the plurality of audio signals, each of the plurality of audio signals corresponds to either “an audio signal to which amplitude modulation is applied” or “an audio signal to which amplitude modulation is not applied”. Similarly, when an input audio signal is composed of a plurality of audio signals and it is determined whether or not amplitude modulation is applied to each of the plurality of audio signals, each of the plurality of audio signals corresponds to either “an audio signal to which amplitude modulation is applied” or “an audio signal to which amplitude modulation is not applied”. In these cases, the function or modulation degree used in amplitude modulation may be different for each of the audio signals to which amplitude modulation is applied, and therefore, the modulation information may include information on “the function used in amplitude modulation” or “the modulation degree” of each audio signal.

[0131] The feature amount information is information related to the feature amount of the audio signal to which amplitude modulation is applied in order to generate the output audio signal included in the content information. For example, the feature amount information can include information related to at least one of the following.

[0132] The distribution of amplitude, energy, or effective value in the frequency domain (e.g., the result of a fast Fourier transform (FFT))

[0133] Distribution of amplitude, energy or effective value in the time domain

[0134] Type of sound source (e.g., vocalization, instrument, object, music, conversational speech, natural sound, or electronic sound)

[0135] Type of sound source for each playback interval

[0136] Content information attributes (e.g., program information for a TV program that includes an audio signal)

[0137] The score is information related to the magnitude of the improvement effect on cognitive function that the user can expect when listening to the sound included in the played content information in an ideal listening environment. Regarding the calculation of the score, as an example, the content information distribution device 30 can refer to a score function for deriving scores corresponding to some or all of the following elements.

[0138] The length of time that the content information (output audio signal) is continuously played (i.e., the listening period)

[0139] The playback volume of content information in an ideal listening environment

[0140] A modulation method that applies amplitude modulation to audio signals

[0141] ·Feature quantity of the audio signal to which amplitude modulation is applied

[0142] The score function may be stored in the storage device 31 or in a storage device provided in an external device (eg, a server) accessible to the content information distribution device 30 .

[0143] Here, the score is defined as the degree of improvement in cognitive function and is related to the estimated result of the amount of brainwaves generated by sound stimulation (i.e., to what extent gamma waves are induced), the index calculated based on the synchronization of brainwaves at each moment (i.e., to what extent the synchronization of gamma waves is achieved) or the index calculated based on the ratio of brainwaves generated at each moment (i.e., to what extent the historical record of gamma waves induced compared to the frequency bands of other brainwaves). As one example, the score is determined to increase (e.g., increase linearly) relative to the increase in the amount of brainwaves generated. As another example, the score is determined to increase almost without increasing within a range where the amount of brainwaves generated is less than a first threshold, and increase relative to the increase in the amount of brainwaves generated after the amount of brainwaves generated exceeds the first threshold. As another example, the score is determined to increase within a range where the amount of brainwaves generated is less than a second threshold, and increase almost without increasing (i.e., saturation) after the amount of brainwaves generated exceeds the second threshold. By using an electroencephalogram to measure the amount of brain waves generated by a subject when the subject listens to a variety of audio signals under a variety of conditions, it is possible to describe the relationship between the elements contained in the meta-information and the amount of brain waves generated. Based on the relationship between the elements contained in the meta-information and the amount of brain waves generated, and the relationship between the score and the amount of brain waves generated, a fractional function can be created for deriving the score corresponding to the element contained in the meta-information. The fractional function can be defined as a formula with the aforementioned elements as independent variables, or as a lookup table.

[0144] As an example, the content information distribution device 30 uses Figure 6 The score function shown is used to calculate the score. Figure 6 In the score function of , the score is defined as increasing or maintaining (in other words, not decreasing) relative to the increase in the listening period. Figure 6 In the score function, the score in the case of playback volume C1 is defined as being higher than the score in the case of playback volume C2 (smaller than C1).

[0145] The content information distribution device 30 may also determine the details of the modulation process performed in step S131 so that the score or the increment of the score is equal to or greater than the reference value. As an example, when the score calculated in step S132 is lower than the reference value or the cumulative score value estimated based on the current increment of the score is lower than the reference value, the content information distribution device 30 may change the modulation setting and perform step S131 again. In addition, the content information distribution device 30 may also perform modulation of the audio signal (S131) ​​after performing the generation of meta-information (S132). In this case, for example, after generating meta-information indicating a specified score, the content information distribution device 30 may determine the details of the modulation process so that the score of the modulated content information is equal to or greater than the score, and perform the modulation process on the content information.

[0146] After step S132, the content information distribution device 30 performs distribution of content information (S133).

[0147] Specifically, the content information distribution device 30 distributes the content information included in the output audio signal generated in step S131 together with the meta-information generated in step S132 to the content information playback device 10. As an example, the content information distribution device 30 may also generate a content information file in a prescribed format by multiplexing the output audio signal and the meta-information. Alternatively, the content information distribution device 30 may also distribute the content information file and the meta-information through different transmission paths. For example, the content information distribution device 30 may also distribute the content information file through ordinary broadcasting and distribute the meta-information through data broadcasting or the Internet.

[0148] In addition, the content information distribution device 30 may also include an unprocessed audio signal in the content information file. The unprocessed audio signal and the output audio signal may be the same except for the presence or absence of amplitude modulation. The viewer of the content information can choose to listen to the sound based on the unprocessed audio signal and the sound based on the output audio signal. For example, the content information distribution device 30 may also distribute the unprocessed audio signal as the voice data of the main voice in the TV program, and distribute the output audio signal as the voice data of the secondary voice. In addition, for example, the content information distribution device 30 may also distribute the unprocessed audio signal as the voice data of the first sub-channel of the TV program arranged in a multi-channel format, and distribute the output audio signal as the voice data of the second sub-channel. In this case, the viewer of the TV can easily choose whether to play the unprocessed audio signal or the output audio signal by switching between the main voice and the secondary voice or switching the audio-visual channel. In addition, the content information distribution device 30 may also include an image signal (a moving image signal or a still image signal) in the content information file. The content information distribution device 30 may distribute a plurality of content information including the same image signal and different audio signals (eg, an output audio signal and an unprocessed audio signal). Furthermore, the content information distribution device 30 may perform encoding (compression) processing before multiplexing various information.

[0149] The content information distribution device 30 ends with step S133. Figure 5 Content information distribution processing.

[0150] In addition, the content information distribution device 30 can uniformly distribute the input audio signal with a certain playing period (such as the music content information of a song) Figure 5 The content information distribution process may also be repeated at each playback interval (e.g., every 100 ms) of the input audio signal. Figure 5 Alternatively, the content information distribution device 30 may continuously perform modulation processing on the input audio signal, such as modulation based on analog signal processing, and output the modulated audio signal. Figure 5 The content information distribution processing shown can also be terminated according to specific termination conditions (for example, a certain period of time has passed, the producer or provider of the content information or the user of the content information playback device 10 has performed an operation, or the output history of the modulated sound has reached a specified state).

[0151] (3-2) Content Information Playback Processing

[0152] The content information playback process according to this embodiment will be described. Figure 7 This is a flowchart of the content information playback process of this embodiment. Figure 8This is a diagram showing an example of a screen displayed during the content information playback process according to the present embodiment. Fig. 9 This is a diagram showing an example of a screen displayed during the content information playback process according to the present embodiment.

[0153] Starts in response to any of the following start conditions being met Figure 7 Content information playback processing.

[0154] ·Called out due to other processing or external instructions Figure 7 Content information playback processing.

[0155] ·The user has performed a Figure 7 The operation of content information playback processing.

[0156] The content information playback device 10 is in a predetermined state (for example, the power is turned on).

[0157] ·The specified date and time has arrived.

[0158] · From a predetermined event (for example, activation of the content information playback device 10, or Figure 7 A specified time has passed since the last execution of the content information playback process.

[0159] like Figure 7 As shown, the content information playback device 10 acquires content information and meta information (S110).

[0160] Specifically, the content information playback device 10 acquires the content information and meta-information distributed by the content information distribution device 30 .

[0161] In the following description, it is assumed that the content information includes an unprocessed audio signal, an output audio signal, and a moving image or still image signal. The content information playback device 10 can select either the unprocessed audio signal or the output audio signal, and play a sound based on the selected audio signal. In addition, the audio mode described later refers to, for example, the playback method of the content information playback device 10 playing audio. In the following description, the mode for playing the sound based on the unprocessed audio signal is set to "unprocessed sound mode", and the mode for playing the sound based on the output audio signal is set to "processed sound mode". However, the voice mode is not limited to this example, and can be defined arbitrarily.

[0162] After step S110 , when the raw sound mode is selected, the content information playback device 10 plays the raw content information ( S111 ).

[0163] Specifically, the content information playback device 10 plays the raw content information among the content information acquired in step S110. Here, the content information playback device 10 may demultiplex the content information (file), decode the encoded data, or a combination thereof.

[0164] As an example, the content information playback device 10 outputs sound based on the unprocessed audio signal through the audio output device 22, and displays a moving image or a still image on the display 21. Thus, the user can listen to the sound based on the unprocessed audio signal while browsing the moving image or still image synchronized with the sound. Figure 8 The screen SC10 is displayed on the display 21.

[0165] The screen SC10 includes an object A10a.

[0166] The object A10a displays information related to the content information being played. As an example, the object A10a displays at least one of the following.

[0167] Information used to identify the content information being played (e.g. content information name, channel name)

[0168] Information indicating that the audio mode is the raw sound mode

[0169] ·The status of the user's score

[0170] Object A10a may be displayed all the time, or may be displayed only for a specified period of time after being displayed according to a specified trigger (e.g., switching of audio channels, switching of voice modes, or switching of multiple content information being played continuously). In addition, the method of presenting information related to the content information being played is not limited to Figure 8 For example, the content information playback device 10 may also present information related to the content information being played to the user through voice broadcasting or lighting of an LED.

[0171] In addition, when displaying the status of the user's score, the content information playback device 10 records the user's audio-visual history record of the content information, and refers to the record to calculate the status of the user's score at any time point. As an example, the content information playback device 10 can also calculate the change of the cumulative score during the object period (that is, the change of the cumulative score over time) by accumulating and adding the scores corresponding to the multiple audio-visual history records. Alternatively, when the user's target score during the object period is determined, the content information playback device 10 can also calculate the ratio of the cumulative score to the target score as the user's achievement or non-achievement. The target score is equivalent to the benchmark of the score that the user should obtain during the object period (that is, the sound that should be listened to) in order to improve cognitive function. The content information playback device 10 can calculate the achievement at a point in time, and can also calculate the change of the achievement during the object period (that is, the change of the achievement over time).

[0172] After step S110 , when the processed audio mode is selected, the content information playback device 10 plays the processed content information ( S112 ).

[0173] Specifically, the content information playback device 10 plays the processed content information among the content information acquired in step S110. Here, the content information playback device 10 may demultiplex the content information (file) (divide the multiplexed content information), decode the encoded data, or a combination thereof.

[0174] As an example, the content information playback device 10 outputs sound based on the output audio signal through the audio output device 22, and displays a moving image or a still image on the display 21. Thus, the user can listen to the sound based on the output audio signal while browsing the moving image or still image synchronized with the sound. Fig. 9 The screen SC11 is displayed on the display 21.

[0175] The screen SC11 includes an object A11a.

[0176] The object A11a displays information related to the content information being played. As an example, the object A11a displays at least one of the following.

[0177] Information used to identify the content information being played (e.g. content information name, channel name)

[0178] Information indicating that the audio mode is the processed sound mode

[0179] Meta information of content information (e.g., score corresponding to content information)

[0180] ·The status of the user's score

[0181] The object A11a may be displayed all the time, or may be displayed only for a specified period of time after being displayed according to a specified trigger (e.g., switching of audio channels, switching of voice modes, or switching of multiple content information played continuously). In addition, the meta-information of the content information may be presented as information different from the content information, such as a program table (including EPG (electronic program guide)) or data broadcasting.

[0182] When the content information playback device 10 detects switching to the processed sound mode during playback of the unprocessed content information ( S111 ), it switches the mode ( S113 ).

[0183] For example, when the input device receives an instruction from the user to switch to the processed audio mode, the content information playback apparatus 10 switches to the processed audio mode.

[0184] Similarly, when the content information playback device 10 detects switching to the unprocessed audio mode during playback of the processed content information ( S112 ), it switches the mode ( S113 ).

[0185] For example, when the input device receives an instruction from the user to switch to the raw sound mode, the content information playback apparatus 10 switches to the raw sound mode.

[0186] For example, the termination is terminated in response to any of the following termination conditions being met: Figure 7 Content information playback processing.

[0187] ·The content information being played has been switched.

[0188] The content information being played has ended.

[0189] ・Terminated due to other processing or external instructions Figure 7 Content information playback processing.

[0190] ·The user performed a Figure 7 The operation of content information playback processing.

[0191] The content information playback device 10 enters a predetermined state (for example, power is lost).

[0192] ·The specified date and time has arrived.

[0193] · From a predetermined event (for example, activation of the content information playback device 10, or Figure 7 A specified time has passed since the start of the content information playback process.

[0194] (4) Summary

[0195] As described above, the content information distribution device 30 of this embodiment generates an audio signal having an amplitude change corresponding to the frequency of the gamma wave by performing modulation processing including amplitude modulation on the audio signal (that is, outputting an audio signal), and outputs the content information including the audio signal together with the corresponding meta-information. The user of the content information playback device 10 can easily determine the content information to be played or the viewing method of the content information by referring to the output meta-information.

[0196] The content information distribution device 30 may modulate the audio signal included in the content information so that the score corresponding to the content information becomes equal to or higher than the reference value. Thus, the content information playback device 10 can obtain a score equal to or higher than the reference value by viewing and listening to the played content information without being particularly aware of the played content information.

[0197] The content information playback device 10 of this embodiment obtains meta-information corresponding to the content information including an audio signal (that is, an output audio signal), and performs control related to the playback of the content information based on the meta-information, and the audio signal has a change in amplitude corresponding to the frequency of the gamma wave. As a result, even if the user has no special knowledge, he can grasp the meta-information of the content information, or watch and listen to the processed content information. In addition, as the above-mentioned control, the content information playback device 10 can also make the screen display of the image included in the content information based on the information of the meta-information corresponding to the content information. As a result, the user can easily grasp the meta-information of the content information.

[0198] The meta-information may also include at least one of information indicating application / non-application of modulation processing, information indicating details of modulation processing, and information indicating a score determined based on the details of modulation processing. This makes it possible to perform appropriate playback processing on the content information or to understand the cognitive function improvement effect that the user can expect by viewing and audio-visual processing of the content information.

[0199] The output audio signal may have a change in amplitude corresponding to a frequency of 35 Hz to 45 Hz. This can induce gamma waves in the brain of the user of the content information playback device 10, thereby achieving an improvement effect of cognitive function.

[0200] (5) Modification

[0201] Modifications of this embodiment will be described.

[0202] (5-1) Modification 1

[0203] Modification 1 will be described. Modification 1 is an example in which the content information playback device modulates an audio signal.

[0204] (5-1-1) Information Processing

[0205] The information processing of Modification 1 is described.

[0206] (5-1-1-1) Content Information Distribution Processing

[0207] The content information distribution process according to Modification 1 is described. Fig.10 This is a flowchart of the content information distribution process of Modification Example 1.

[0208] Fig.10 The content information distribution process is implemented by the processor 32 of the content information distribution device 30 reading out and executing the program stored in the storage device 31 .

[0209] The content information distribution process of the present embodiment can be started in response to the same start condition being satisfied. Fig.10 Content information distribution processing.

[0210] like Fig.10 As shown, the content information distribution device 30 and Figure 6 The acquisition of the input audio signal is similarly performed (S130).

[0211] After step S131, the content information distribution device 30 performs generation of meta information (S232).

[0212] Specifically, the content information distribution device 30 generates meta-information corresponding to the modulation process recommended to be performed in the content information playback device 10. The meta-information can include the information described in this embodiment. The modulation process recommended to be performed in the content information playback device 10 can be determined based on the input operation of the producer or provider of the content information corresponding to the input audio signal, or an instruction from the outside, or can be determined by an algorithm. However, the content information distribution device 30 can also perform the modulation process based on the modulation information included in the meta-information to calculate the score.

[0213] After step S232, the content information distribution device 30 performs distribution of content information (S233).

[0214] Specifically, the content information distribution device 30 distributes the content information based on the input audio signal acquired in step S130 (however, the processed content information may not be included) together with the meta-information generated in step S232 to the content information playback device 10. As an example, the content information distribution device 30 may also generate a content information file of a specified format by multiplexing an unprocessed audio signal (such as an input audio signal) and meta-information. Viewers of the content information can choose to listen to either the sound based on the unprocessed audio signal or the sound based on the audio signal amplitude-modulated by the content information playback device 10. In addition, the content information distribution device 30 may also include an image signal (a moving image signal or a still image signal) in the content information file. Furthermore, the content information distribution device 30 may also perform encoding (compression) processing before multiplexing various information.

[0215] The content information distribution device 30 ends with step S233. Fig.10 In addition, the content information distribution device 30 can uniformly perform processing on the input audio signal with a certain playback period (such as the music content information of a song). Fig.10 The content information distribution process may also be repeated at each playback interval (e.g., every 100 ms) of the input audio signal. Fig.10 Alternatively, the content information distribution device 30 may continuously perform modulation processing on the input audio signal, such as modulation based on analog signal processing, and output the modulated audio signal. Fig.10 The content information distribution processing shown can also be terminated according to specific termination conditions (for example, a certain period of time has passed, the producer or provider of the content information or the user of the content information playback device 10 has performed an operation, or the output history of the modulated sound has reached a specified state).

[0216] (5-1-1-2) Content Information Playback Processing

[0217] The content information playback process according to Modification 1 is described. Fig.11 This is a flowchart of the content information playback process of Modification Example 1.

[0218] Can respond to the content information playback processing ( Figure 7 ) The same starting conditions are met and the Fig.11 Content information playback processing.

[0219] like Fig.11 As shown, the content information playback device 10 and Figure 7 The acquisition of content information and meta information is similarly performed ( S110 ).

[0220] In the following description, it is assumed that the content information includes an unprocessed audio signal and a moving image or still image signal. The content information playback device 10 can select either the unprocessed audio signal or an audio signal (output audio signal) obtained by amplitude modulating at least a portion of the audio signal, and play a sound based on the selected audio signal.

[0221] After step S110, when the raw audio mode is selected, the content information playback device 10 and Figure 7 The raw content information is played back in the same manner (S111).

[0222] After step S110 , when the processed sound mode is selected, the content information playback device 10 modulates the audio signal ( S212 ).

[0223] Specifically, the content information playback device 10 generates an output audio signal by at least partially amplitude modulating the raw audio signal included in the content information acquired in step S110. Here, the content information playback device 10 may also demultiplex the content information (file), decode the encoded data, or a combination thereof.

[0224] As a first example of audio signal modulation (S212), the content information playback device 10 generates an intermediate audio signal based on the input audio signal which is a raw audio signal. The content information playback device 10 may determine a method of generating the intermediate audio signal by referring to the meta-information acquired in step S110.

[0225] The content information playback device 10 selects a target signal from a plurality of audio signals included in the intermediate audio signal. The content information playback device 10 may also refer to the meta-information to determine which audio signal to select as the target signal.

[0226] The content information playback device 10 performs amplitude modulation on the selected target signal. The content information playback device 10 may determine at least one of the modulation function and the modulation degree by referring to the meta-information.

[0227] The content information playback device 10 generates an output audio signal based on the non-target signal and the modulated target signal. The technique for generating the output audio signal based on the non-target signal and the modulated target signal may be the same as that of the present embodiment.

[0228] In the second example of the modulation of the audio signal (S212), the raw audio signal includes a plurality of audio signals having different characteristics. The content information playback device 10 selects one or more audio signals to which amplitude modulation is to be applied from the plurality of audio signals included in the raw audio signal. The second example of the modulation of the audio signal (S212) can be understood by appropriately replacing the "intermediate audio signal" in the description of the first example with the "raw audio signal".

[0229] In the third example of audio signal modulation (S212), the content information playback device 10 amplitude modulates the raw audio signal. The amplitude modulation of the raw audio signal is the same as the amplitude modulation of the target signal described in the first example of audio signal modulation (S212).

[0230] The content information playback device 10 generates an output audio signal based on the modulated audio signal. The technique for generating the output audio signal based on the modulated audio signal may be the same as that of the present embodiment.

[0231] After step S212, the content information playback device 10 plays the processed content information (S213).

[0232] Specifically, the content information playback device 10 plays the output audio signal (processed content information) generated in step S212.

[0233] As an example, the content information playback device 10 outputs sound based on the output audio signal through the audio output device 22, and displays a moving image or a still image on the display 21. Thus, the user can listen to the sound based on the output audio signal while browsing the moving image or still image synchronized with the sound. Fig. 9 The screen SC11 is displayed on the display 21.

[0234] When the content information playback device 10 detects switching to the processed sound mode during playback of the unprocessed content information ( S111 ), it performs modulation of the audio signal ( S212 ).

[0235] For example, when the input device receives an instruction from the user to switch to the processed sound mode, the content information playback apparatus 10 detects the switch to the processed sound mode.

[0236] Similarly, when the content information playback device 10 detects switching to the raw sound mode during modulation of the audio signal ( S212 ) or playback of the processed content information ( S213 ), it plays the raw content information ( S111 ).

[0237] For example, when the input device receives an instruction from the user to switch to the raw sound mode, the content information playback apparatus 10 detects the switching to the raw sound mode.

[0238] The content information playback process of the present embodiment can be terminated in response to the same termination condition being satisfied. Fig.11 Content information playback processing.

[0239] (5-1-2) Summary

[0240] As described above, the content information playback device 10 of the modification example 1 modulates the audio signal included in the content information distributed from the content information distribution device 30 based on the modulation information included in the meta-information. Thus, even if the content information distribution device 30 does not distribute the processed content information, the user of the content information playback device 10 can view and listen to the processed content information. Therefore, the modification example 1 is useful, for example, when the capacity of the transmission path for distributing the content information is limited.

[0241] In addition, in the description of the above-mentioned modification example 1, an example is shown in which the content information playback device 10 dynamically switches the modulation process during the playback of the content information. However, the present invention is not limited thereto, and the content information playback device 10 may also perform the modulation process in units of content information based on the meta-information. For example, the content information playback device 10 may also determine the details of the modulation process performed in step S212 so that the score of the modulated content information becomes equal to or greater than the reference value.

[0242] (5-2) Modification 2

[0243] Modification 2 will be described. Modification 2 is an example in which the content information playback device 10 controls content information recommended to the user and a mode when the content information is played.

[0244] (5-2-1) Content Information Playback Processing

[0245] The content information playback process according to Modification 2 is described. Fig.12 This is a flowchart of the content information playback process of Modification Example 2.

[0246] Can respond to the content information playback processing ( Figure 7 ) The same starting conditions are met and the Fig.12 Content information playback processing.

[0247] like Fig.12 As shown, the content information playback device 10 acquires meta information (S310).

[0248] Specifically, the content information playback device 10 acquires meta-information of a plurality of playable content information. As a first example of acquisition of meta-information (S310), the content information playback device 10 acquires meta-information distributed by the content information distribution device 30. As a second example of acquisition of meta-information (S310), the content information playback device 10 acquires meta-information by referring to an information source (e.g., EPG) storing meta-information of content information. As a third example of acquisition of meta-information (S310), the content information playback device 10 acquires meta-information by demultiplexing content information stored in the storage device 11. As a fourth example of acquisition of meta-information (S310), the content information playback device 10 acquires meta-information input by a user.

[0249] After step S310, the content information playback device 10 performs selection of content information and mode (S311).

[0250] Specifically, the content information playback device 10 selects content information recommended to the user and a mode for playing the content information based on the meta-information acquired in step S310. As an example, the content information playback device 10 may select content information or a mode based on at least one of the following user information.

[0251] User attributes (e.g. age, gender, birthplace, location, language used)

[0252] ・Users’ operation history (e.g. browsing history of websites for the elderly, operation history of users suspected of cognitive impairment)

[0253] The user's viewing history of content information (for example, whether the user has fully viewed content information including sound based on the output audio signal on the current day)

[0254] · Results of user survey responses

[0255] Results of the user's cognitive impairment test

[0256] The content information playback device 10 can also estimate the user's preferences (e.g., genre, artist, etc.) based on such user information, and can select content information or mode according to the preferences. For example, the content information playback device 10 can also give priority to the content information preferred by the user, or suppress the frequency of playing the content information preferred by the user in the processed sound mode.

[0257] In addition, the content information playback device 10 may select content information and a mode based on the scores of each content information so that the user can achieve the score acquisition goal. For example, the content information playback device 10 may preferentially select content information with a high score or content information with a score exceeding a predetermined threshold.

[0258] In step S311, the content information playback device 10 may select only one piece of content information that the user is watching immediately and the mode when playing the content information, or may select a combination of multiple content information that the user is watching in the future (that is, a playlist) and the mode when playing the content information. For example, the content information playback device 10 may create a playlist in such a way that the total value of the scores of the content information included in the playlist is greater than a reference value. The reference value may be determined for each user or may be shared by multiple users. The content information playback device 10 may obtain the reference value from an external device, read the reference value from the storage device 11, or determine the reference value based on user input. In addition, the selection of content information and mode in S311 may be performed automatically by the content information playback device 10 based on meta-information, or may be performed based on a user operation on a candidate recommended by the content information playback device 10 based on meta-information.

[0259] After step S311, the content information playback device 10 acquires the content information (S312).

[0260] Specifically, the content information playback device 10 acquires the content information selected in step S311. As a first example of acquiring content information (S312), the content information playback device 10 acquires content information distributed by the content information distribution device 30. As a second example of acquiring content information (S312), the content information playback device 10 acquires content information stored in the storage device 11.

[0261] After step S312, when the raw sound mode is selected, the content information playback device 10 plays the raw content information (S313).

[0262] Specifically, the content information playback device 10 plays the raw content information in the content information acquired in step S312. Here, the content information playback device 10 may demultiplex the content information (file), decode the encoded data, or a combination thereof.

[0263] As an example, the content information playback device 10 outputs sound based on the unprocessed audio signal through the audio output device 22, and displays a moving image or a still image on the display 21. Thus, the user can listen to the sound based on the unprocessed audio signal while browsing the moving image or still image synchronized with the sound. Figure 8 The screen SC10 is displayed on the display 21.

[0264] After step S312, when the processed sound mode is selected, the content information playback device 10 modulates the audio signal (S314).

[0265] Specifically, the content information playback device 10 generates an output audio signal by at least partially performing amplitude modulation on the raw audio signal included in the content information acquired in step S312. Here, the content information playback device 10 may also perform demultiplexing of the content information (file), decoding of the encoded data, or a combination thereof. However, in the case where the content information includes the output audio signal, the content information playback device 10 may omit the modulation of the audio signal (S314).

[0266] As a first example of modulation of the audio signal (S314), the content information playback device 10 generates an intermediate audio signal based on the raw audio signal, that is, the input audio signal. The content information playback device 10 may also determine a method for generating the intermediate audio signal by referring to the meta-information acquired in step S310 (meta-information corresponding to the content information acquired in step S312).

[0267] The content information playback device 10 selects a target signal from a plurality of audio signals included in the intermediate audio signal. The content information playback device 10 may also refer to the meta-information to determine which audio signal to select as the target signal.

[0268] The content information playback device 10 performs amplitude modulation on the selected target signal. The content information playback device 10 may determine at least one of the modulation function and the modulation degree by referring to the meta-information.

[0269] The content information playback device 10 generates an output audio signal based on the non-target signal and the modulated target signal. The technique for generating the output audio signal based on the non-target signal and the modulated target signal may be the same as that of the present embodiment.

[0270] In the second example of the modulation of the audio signal (S314), the raw audio signal includes a plurality of audio signals having different characteristics. The content information playback device 10 selects one or more audio signals to which amplitude modulation is to be applied from the plurality of audio signals included in the raw audio signal. The second example of the modulation of the audio signal (S314) can be understood by appropriately replacing the "intermediate audio signal" in the description of the first example with the "raw audio signal".

[0271] In the third example of audio signal modulation (S314), the content information playback device 10 amplitude modulates the raw audio signal. The amplitude modulation of the raw audio signal is the same as the amplitude modulation of the target signal described in the first example of audio signal modulation (S314).

[0272] The content information playback device 10 generates an output audio signal based on the modulated audio signal. The technique for generating the output audio signal based on the modulated audio signal may be the same as that of the present embodiment.

[0273] After step S314, the content information playback device 10 plays the processed content information (S315).

[0274] Specifically, the content information playback device 10 plays the output audio signal (processed content information) generated in step S314. However, when the content information acquired in step S312 includes the output audio signal, the content information playback device 10 can play the output audio signal included in the content information.

[0275] As an example, the content information playback device 10 outputs sound based on the output audio signal through the audio output device 22, and displays a moving image or a still image on the display 21. Thus, the user can listen to the sound based on the output audio signal while browsing the moving image or still image synchronized with the sound. Fig. 9 The screen SC11 is displayed on the display 21.

[0276] The content information playback device 10 repeats the processing of steps S312 to S313 or steps S312, S314 to S315 until the playback of the content information selected in step S311 is completed. When the playback of the content information selected in step S311 is completed, the content information playback device 10 may execute the acquisition of meta information (S310) or the selection of content information and mode (S311) again, or may terminate the playback. Fig.12 Content information playback processing.

[0277] The content information playback process of the present embodiment can be terminated in response to the same termination condition being satisfied. Fig.12 Content information playback processing.

[0278] (5-2-2) Summary

[0279] As described above, the content information playback device 10 of Modification 2 selects content information recommended to the user based on the meta-information of the content information and the user information. Thus, the user can watch content information that is appropriate from the perspective of preference and cognitive function improvement effect without actively selecting the content information to be played.

[0280] In addition, the content information playback device 10 may generate a list in such a manner that the total score of the content information included in the list of one or more content information to be played (play list) is greater than or equal to a reference value. Thus, even if the user does not actively select the content information to be played, he or she can view and listen to the content information group that can obtain a score greater than the reference value.

[0281] (7) Other Modifications

[0282] The storage device 11 may be connected to the content information playback device 10 via a network. The display 21 may be integrated with the content information playback device 10. The storage device 31 may be connected to the content information distribution device 30 via a network.

[0283] Each step of the above-described information processing can be executed in any one of the content information playback device 10 and the content information distribution device 30. Part of the above-described information processing steps may be executed by an external device (eg, a server) not shown.

[0284] In the above description, an example is shown in which the content information playback device 10 is connected to one audio output device 22. The content information playback device 10 may be connected to a plurality of audio output devices 22, and in this case, the content information playback device 10 may be configured to select which audio output device 22 to send an audio signal to.

[0285] The content information playback device 10 may be provided with a function of conducting a questionnaire survey or a cognitive function test (for example, a function of measuring cognitive function by conducting a voice conversation with the user).

[0286] The algorithm for determining the method for generating the output audio signal (e.g., the modulation method, the synthesis ratio of two or more audio signals, the method for determining the audio signal to which amplitude modulation is applied, etc.) or the default method for generating the output audio signal may be changed by updating the firmware of the content information distribution device 30. For example, the firmware may be updated by the content information distribution device 30 communicating with an external device (e.g., a server not shown).

[0287] Although an example of acquiring the input audio signal from the sound source device 41 is shown, the input audio signal may be generated by receiving the ambient sound through a microphone. The input audio signal may also be read from the storage device 31 or a removable medium connected to the content information distribution device 30. The sound source device 41 (including the sound source device 41 built into the content information distribution device 30) may also generate the input audio signal at any time.

[0288] In the above description, an example of modulating an audio signal derived from an input audio signal is shown, but an output audio signal may be generated based on the input audio signal and a modulated audio signal that is not derived from the input audio signal.

[0289] In the above description, the example in which the modulation function has a frequency of 35 Hz or more and 45 Hz or less is mainly used for explanation. However, the modulation function used by the content information playback device 10 or the content information distribution device 30 is not limited to this, as long as it is a modulation function that affects the induction of gamma waves in the listener's brain. For example, the modulation function may also have a frequency of 25 Hz or more and 140 Hz or less. In addition, for example, the frequency of the modulation function may change over time, and the modulation function may partially have a frequency less than 35 Hz or a frequency higher than 45 Hz.

[0290] In the above description, an example of distributing content information together with meta-information is described. However, instead of distributing content information together with meta-information, they can also be recorded in a storage medium. In this case, the content information distribution device can be replaced by a content information recording device. The content information playback device can read the content information and meta-information from the storage medium, and play the content information and perform control based on the meta-information.

[0291] In the above description, the example in which the content information distribution device 30 generates meta-information is shown, but the present invention is not limited to this. For example, the content information producer, broadcasting operator, or viewer of the content information may set the meta-information. Alternatively, the set meta-information may be stored in a storage device or recording medium, and the content information distribution device 30 may obtain the meta-information from an external storage device or storage medium. In this case, the content information distribution device 30 may obtain the content information and the meta-information separately, or may obtain the content information to which the meta-information is attached. In addition, for example, the content information distribution device 30 may determine the meta-information based on the user's input.

[0292] In addition, the meta-information may not be distributed to the content information playback device 10. For example, the meta-information may be published on a web page, newspaper, magazine, etc., or may be notified to a terminal used by a user (for example, a notification of an application or an email notification). In this case, the content information playback device 10 may obtain the meta-information by, for example, referring to an information source storing the meta-information of the content information, or may obtain the meta-information input by the user.

[0293] Furthermore, when the content information distribution device 30 acquires additional information (eg, an ID3 tag in an MP3 file) together with the input audio signal, it may also output the additional information to an external device together with the output audio signal after modifying the additional information.

[0294] In the present embodiment, an example is shown in which the content information distribution device 30 distributes content information including an unprocessed audio signal and an output audio signal. Here, the number of multiplexed audio signals of the distributed content information can also be arbitrarily determined. That is, the output audio signals to which different modulation processes (for example, modulation processes in which at least one of the modulation function and the modulation degree is different) are applied can also be multiplexed and distributed. In addition, in addition to the output audio signal and the unprocessed audio signal, the content information distribution device 30 can also multiplex and distribute other audio signals. For example, the content information distribution device 30 can also multiplex and distribute the unprocessed sound of the first language, the processed sound of the first language, the unprocessed sound of the second language, and the processed sound of the second language.

[0295] It is also possible to synthesize an audio signal (e.g., an audio signal corresponding to a buzzer, chirp, noise, or BGM) that has been amplitude modulated during a period corresponding to a silent portion in an unprocessed audio signal. This can improve the cognitive function of the user of the content information playback device 10 without hindering the listening of the original sound of the content information. Such synthesis of audio signals can be performed by either the content information distribution device 30 or the content information playback device 10.

[0296] In variation 1, an example of modulation processing performed by the content information playback device 10 is shown. The content information playback device 10 may perform modulation processing according to the modulation information included in the meta-information, or may perform modulation processing according to the user's instructions. As an example, the user may also select a modulation setting (e.g., a modulation function or a modulation degree). In this case, the content information playback device 10 may also present information on the recommended modulation setting to the user based on the modulation information included in the meta-information.

[0297] In the second modification, an example is described in which the content information playback device 10 selects content information recommended to the user and a mode for playing the content information, and automatically controls the playback of the content information according to the selection result. However, the user may be able to change the content information or mode selected by the content information playback device 10. Thus, the user can refer to the selection result of the content information playback device 10 as a judgment material and decide the content information to be actually viewed or the mode for viewing the content information based on his or her own intention or preference.

[0298] The embodiments of the present invention have been described in detail above, but the scope of the present invention is not limited to the above embodiments. In addition, the above embodiments can be variously improved and changed within the scope of the gist of the present invention. In addition, the above embodiments and modifications can be combined.

[0299] Description of Reference Numerals

[0300] 1: cognitive function improvement system; 10: content information playback device; 11: storage device; 12: processor; 13: input and output interface; 14: communication interface; 21: display; 22: audio output device; 30: content information distribution device; 31: storage device; 32: processor; 33: input and output interface; 34: communication interface; 41: sound source device.

Claims

1. A signal processing device, comprising: A receiving unit, which receives an input audio signal; a generating unit that generates a modulated audio signal having an amplitude change corresponding to a frequency of a gamma wave by performing a modulation process including amplitude modulation on the input audio signal received by the receiving unit; as well as The output unit outputs the content information including the modulated audio signal generated by the generating unit and the modulation information related to the modulation process performed to generate the modulated audio signal in a corresponding manner.

2. The signal processing device according to claim 1, wherein: further comprising a reference value acquisition unit for acquiring a reference value of a score determined based on the content of the modulation process, The generating unit modulates the input audio signal received by the receiving unit so that a score determined based on the content of the modulation process performed to generate a modulated audio signal from the input audio signal becomes equal to or greater than the reference value acquired by the reference value acquiring unit.

3. A signal processing device, comprising: a modulation information acquisition unit that acquires modulation information corresponding to a modulated audio signal and related to a modulation process including amplitude modulation performed to generate the modulated audio signal, the modulated audio signal having a change in amplitude corresponding to a frequency of a gamma wave; and A control unit performs control related to the playback of content information including a modulated audio signal corresponding to the modulation information based on the modulation information acquired by the modulation information acquisition unit.

4. The signal processing device according to claim 3, wherein: The control performed by the control unit includes control for causing a screen displaying an image included in the reproduced content information to display modulation information related to a modulation process performed to generate a modulated audio signal included in the content information.

5. The signal processing device according to claim 3, wherein: The device further comprises a user information acquisition unit for acquiring user information related to at least any one of a user's attributes, an operation history, and a viewing history, The control performed by the control unit includes control for prompting the user to play content information determined based on the user information acquired by the user information acquisition unit and the modulation information acquired by the modulation information acquisition unit.

6. The signal processing device according to claim 3, wherein: further comprising a reference value acquisition unit for acquiring a reference value of a score determined based on the content of the modulation process, The control performed by the control unit includes the control of generating a list of one or more content information as playback objects, in which the list is generated in such a way that the sum of scores determined based on the content of the modulation processing performed to generate the modulated audio signal included in the content information in the list becomes greater than the reference value acquired by the reference value acquisition unit.

7. The signal processing device according to any one of claims 1 to 6, wherein: The modulation information includes at least any one of information indicating that a modulation process has been performed, information indicating the content of the modulation process, and information indicating a score determined based on the content of the modulation process.

8. The signal processing device according to any one of claims 1 to 6, wherein: The modulated audio signal has a change in amplitude corresponding to a frequency greater than or equal to 35 Hz and less than or equal to 45 Hz.

9. A cognitive function improvement system, comprising a content information distribution device and a content information playback device, wherein: The content information distribution device comprises: A receiving unit, which receives an input audio signal; a generating unit that generates a modulated audio signal having an amplitude change corresponding to a frequency of a gamma wave by performing a modulation process including amplitude modulation on the input audio signal received by the receiving unit; as well as an output unit configured to output content information including the modulated audio signal generated by the generating unit and modulation information related to a modulation process performed to generate the modulated audio signal in a corresponding manner, The content information playing device comprises: a modulation information acquisition unit, which acquires the modulation information output by the output unit; as well as A control unit performs control related to the playback of the content information output by the output unit based on the modulation information acquired by the modulation information acquisition unit.

10. A signal processing method, comprising the following processing: Accept input audio signal; generating a modulated audio signal having an amplitude change corresponding to the frequency of a gamma wave by performing a modulation process including amplitude modulation on the received input audio signal; and Content information including the generated modulated audio signal is outputted in association with modulation information related to the modulation process performed to generate the modulated audio signal.

11. A signal processing method, comprising the following processing: Acquiring modulation information corresponding to a modulated audio signal, which is related to a modulation process including amplitude modulation performed to generate the modulated audio signal, the modulated audio signal having a change in amplitude corresponding to a frequency of a gamma wave; and Based on the acquired modulation information, control related to the playback of content information including a modulated audio signal corresponding to the modulation information is performed.

12. A program for causing a computer to execute the signal processing method according to claim 10 or 11.

Images