Audio output apparatus, terminal apparatus, and method for information processing

Abstract

To provide an audio output apparatus, a terminal apparatus, and a method for information processing that determine hearing ability on the basis of a user operation related to output volume.SOLUTION: The audio output apparatus includes: a processing unit for controlling output of audio data; and a storage unit for storing volume operation information in which operation content and operation timing of a user operation are associated with each other in a case where a user operation related to the output volume of the audio data is received, the processing unit determining whether hearing ability of a user is reduced on the basis of the volume operation information acquired in a predetermined period.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an audio output device, a terminal device, an information processing method, and the like. [Background technology]

[0002] Conventionally, devices that output audio in consideration of the ease of hearing for viewers have been known. For example, Patent Document 1 discloses a television viewing system that automatically adjusts audio signals based on the viewer's hearing ability data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-230071 Summary of the Invention [Problem to be solved by the invention]

[0004] The method of Patent Document 1 requires that the hearing ability data of the viewer be registered in advance. According to some aspects of the present disclosure, it is possible to provide an audio output device, a terminal device, an information processing method, and the like that determine hearing ability based on a user operation related to output volume. [Means for solving the problem]

[0005] One aspect of the present disclosure includes a processing unit that controls the output of audio data, and a memory unit that, when a user operation regarding the output volume of the audio data is received, stores volume operation information that corresponds to the operation content and operation timing of the user operation, and the processing unit is related to an audio output device that determines whether the user's hearing has deteriorated based on the volume operation information acquired over a predetermined period of time.

[0006] Another aspect of the present disclosure includes a communication unit that communicates with an audio output device, an operation unit that accepts user operations related to the output volume of audio data in the audio output device, a processing unit that performs processing to transmit a control signal for the output volume to the audio output device via the communication unit based on the user operation, and a memory unit that stores volume operation information that corresponds to the operation content and operation timing of the user operation, wherein the processing unit is related to a terminal device that determines whether the user's hearing has deteriorated based on the volume operation information acquired over a predetermined period of time.

[0007] Yet another aspect of the present disclosure relates to an information processing method that accepts a user operation regarding the output volume of audio data in an audio output device, acquires volume operation information that corresponds to the operation content and operation timing of the user operation, and determines whether the user's hearing has deteriorated based on the volume operation information acquired over a predetermined period of time. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration example of an information processing system including an audio output device. [Figure 2] 1 is a diagram illustrating a configuration example of a display device that is an example of an audio output device. [Figure 3] 1 is a flowchart illustrating processing according to an embodiment of the present invention. [Figure 4A] 10 is an example of volume operation information. [Figure 4B] 10 is an example of volume operation information for a given time period. [Figure 5] 10 is an example of a notification screen displayed on a display unit of a display device. [Figure 6] 1 is a configuration example of an information processing system including an audio output device, a terminal device, and a server system. [Figure 7] 1 is a configuration example of a terminal device. [Figure 8] 3A and 3B are schematic diagrams illustrating the directivity of a plurality of sound output units. [Figure 9] 10 is an example of a notification screen displayed on a display unit of a display device. [Figure 10] 10 is an example of volume operation information acquired at a given timing. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, the present embodiment will be described with reference to the drawings. In the drawings, identical or equivalent elements are designated by the same reference numerals, and duplicate explanations will be omitted. Note that the present embodiment described below does not unduly limit the content described in the claims. Furthermore, not all of the configurations described in the present embodiment are necessarily essential components of the present disclosure.

[0010] 1. System configuration example FIG. 1 is a diagram illustrating an example of the configuration of an information processing system 10 according to this embodiment. The information processing system 10 includes an audio output device 20 and a remote controller 500. However, the configuration of the information processing system 10 is not limited to the example in FIG. 1, and various modifications are possible, such as omitting some components or adding other components. For example, the information processing system 10 may include a terminal device 200 (terminal device 200-1 and / or terminal device 200-2) as described below with reference to FIG. 6, or may include a server system 300. Furthermore, the terminal device 200 may operate as the remote controller 500.

[0011] The audio output device 20 according to this embodiment is, for example, a display device 100 having a display panel as shown in FIG. 1. The display device 100 may be a television receiver. The television receiver is a device that receives, for example, television broadcast waves and displays images based on the received broadcast waves on a display panel (display unit 140). Note that the external appearance of the display device 100 in FIG. 1 is an example, and various modifications can be made to the specific shape. The audio output device 20 may also be a device (for example, a speaker) that does not have a display unit.

[0012] The remote controller 500 is an operation interface including various buttons for operating the display device 100, such as a channel selection button and a volume adjustment button. The remote controller 500 transmits operation contents to the display device 100 using infrared rays. Alternatively, the remote controller 500 may be connected to the display device 100 using short-range wireless communication such as Bluetooth.

[0013] 2 is a block diagram showing a detailed configuration example of a display device 100, which is an example of the audio output device 20. The display device 100 may include, for example, a processing unit 110, a storage unit 120, a communication unit 130, a display unit 140, an operation unit 150, and a first sound output unit 160. These components are connected to each other, for example, via a bus provided in the display device 100. However, the configuration of the display device 100 is not limited to that shown in FIG. 2, and modifications such as omitting some components or adding other components are possible. For example, as described above, the audio output device 20 may have a configuration in which the display unit 140 in FIG. 2 is omitted.

[0014] The processing unit 110 of this embodiment is configured by the following hardware. The hardware can include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the hardware can be configured by one or more circuit devices or one or more circuit elements mounted on a circuit board. The one or more circuit devices are, for example, an integrated circuit (IC), a field-programmable gate array (FPGA), etc. The one or more circuit elements are, for example, a resistor, a capacitor, etc.

[0015] The processing unit 110 may also be implemented by the following processor. The audio output device 20 (display device 100) of this embodiment includes a memory that stores information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various data. The memory may be the storage unit 120 or another memory. The processor includes hardware. Various processors can be used as the processor, such as a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). The memory may be a semiconductor memory such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory, or may be a register, a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk drive. For example, the memory stores computer-readable instructions, and the processor executes the instructions to implement the functions of the processing unit 110 as processing. The instruction here may be an instruction from an instruction set that constitutes a program, or an instruction that instructs the hardware circuitry of the processor to operate.

[0016] The storage unit 120 is a work area for the processing unit 110 and stores various information. The storage unit 120 may store, for example, volume operation information, which will be described later with reference to Figures 4A and 4B. The storage unit 120 can be realized by various types of memory, and the memory may be a semiconductor memory such as an SRAM, a DRAM, a ROM (Read Only Memory), or a flash memory, or may be a register, a magnetic storage device, or an optical storage device.

[0017] The communication unit 130 is an interface for performing communication, and includes, for example, an antenna, an RF (radio frequency) circuit, and a baseband circuit when the display device 100 performs wireless communication. The communication unit 130 may operate under the control of the processing unit 110, or may include a processor for communication control that is different from the processing unit 110.

[0018] The communication unit 130 is connected to, for example, a remote controller 500. The communication unit 130 may perform short-distance wireless communication according to various methods such as Bluetooth. The communication unit 130 may also include an infrared receiving unit that receives an operation signal (infrared signal) from the remote controller 500.

[0019] 2, the communication unit 130 may be connected to a second sound output unit 400 that is provided separately from the display device 100. The communication unit 130 and the second sound output unit 400 may perform wireless communication, or may perform wired communication using various methods such as USB (Universal Serial Bus).

[0020] Furthermore, the communication unit 130 may be connected to the terminal device 200 and the server system 300, as will be described later with reference to Fig. 6. In this case, the communication unit 130 may perform communication according to a method defined in IEEE802.11 or IEEE802.3, or may perform communication according to another method. In this case, the communication unit 130 may be connected to the terminal device 200 and the server system 300 via a communication device (e.g., a router) not shown in Fig. 6 etc.

[0021] The method used for communication between the communication unit 130 and each device is not limited to the above example, and various modifications are possible.

[0022] Display unit 140 is an interface that displays various information such as video data (contents), and may be a liquid crystal display, an organic EL display, or any other type of display.

[0023] The operation unit 150 is an interface that accepts user operations. The operation unit 150 may be a button or the like provided on the display device 100. However, the operation of the display device 100 may be performed only by the remote controller 500, and the operation unit 150 provided on the main body of the display device 100 may be omitted.

[0024] The first sound output unit 160 is a speaker that outputs audio data. For example, the display device 100 may include a tuner (not shown in FIG. 2). The tuner includes an interface that receives television broadcast waves of a specific frequency, a circuit that executes processing on the received broadcast waves, and the like. For example, the processing unit 110 acquires video data and audio data corresponding to the television broadcast waves via the tuner, and performs control to display the video data on the display unit 140 and control to output the audio data to the first sound output unit 160.

[0025] However, the output destination of the audio data is not limited to the first sound output unit 160 (built-in speaker) of the display device 100. For example, as shown in Fig. 2, the processing unit 110 may output the audio data to the second sound output unit 400 (external speaker, connected audio) via the communication unit 130.

[0026] As described above, the audio output device 20 (display device 100) according to this embodiment includes a processing unit 110 and a storage unit 120. The processing unit 110 controls the output of audio data. When a user operation related to the output volume of audio data is received, the storage unit 120 stores volume operation information in which the operation content and operation timing of the user operation are associated. Details of the volume operation information will be described later with reference to FIGS. 4A and 4B. In this case, the user operation is, in a narrow sense, an operation to instruct a volume change using the remote controller 500, and the volume operation information is acquired based on a signal received by the communication unit 130 from the remote controller 500. However, if a volume change button is provided on the main body of the display device 100, the user operation may include an operation on the volume change button.

[0027] The processing unit 110 then determines whether the user's hearing has deteriorated based on the volume operation information acquired during a predetermined period. The predetermined period here is, for example, a period during which the user's hearing may change to a degree that is clearly distinguishable from its previous state, such as a period of about three months. However, the length of the predetermined period is not limited to this, and various modifications are possible.

[0028] According to the method of the present embodiment, a user's hearing loss can be determined by referring to a history of voice operation information acquired over a certain period of time. Volume operation information can be naturally accumulated as the user watches and listens to content on a daily basis using the display device 100. Therefore, hearing loss can be assessed without the user having to consciously perform operations related to the assessment, thereby improving convenience. Furthermore, when a user's hearing loss is detected, a notification process can be performed to make the user or their family aware of the hearing loss. This prevents the output volume from becoming excessively loud, thereby enabling noise reduction. Alternatively, the processing unit 110 may perform a notification process to switch to a speaker with high directionality when hearing loss is detected, as will be described later with reference to FIGS. 8 and 9 . In this case, noise reduction can also be achieved by reducing the likelihood of excessively loud sounds being heard by people other than the target user.

[0029] Furthermore, some or all of the processing performed by the information processing system 10 of this embodiment may be realized by a program. The processing performed by the information processing system 10 is, in a narrow sense, processing performed by the processing unit 110 of the audio output device 20 (display device 100), but may also include processing performed by other devices. For example, the program may be a program that realizes processing performed by the processing unit 210 of the terminal device 200 (described later) or processing performed by a processor of the server system 300.

[0030] The program according to this embodiment can be stored in, for example, a non-transitory information storage medium (information storage device), which is a medium readable by a computer. The information storage medium can be realized by, for example, an optical disc, a memory card, a HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The processing unit 110 and the like perform various processes of this embodiment based on the program stored in the information storage medium. In other words, the information storage medium stores a program for causing a computer to function as the processing unit 110 and the like. A computer is a device equipped with an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to this embodiment is a program for causing a computer to execute each step described below using FIG. 3 and the like.

[0031] Furthermore, the technique of this embodiment can be applied to an information processing method including the steps of: accepting a user operation related to the output volume of audio data in an audio output device; acquiring volume operation information in which the operation content and operation timing of the user operation are associated; and determining whether the user's hearing has deteriorated based on the volume operation information acquired over a predetermined period. Note that the information processing method is, for example, a control method for the audio output device 20 (display device 100), but may also be a control method for the terminal device 200 described later, a control method for the server system 300, or a control method for the information processing system 10 including these.

[0032] 2. Processing Details 3 is a flowchart illustrating processing according to this embodiment. An example of the display device 100 will be described below, but the display device 100 in the following description can be expanded to an audio output device 20. For example, in an embodiment in which the display unit 140 is not essential, the display device 100 in the following description can be replaced with an audio output device 20 that does not have a display unit.

[0033] 3 may be repeatedly executed, for example, while the power of the display device 100 is on. When this process starts, first, in step S101, the processing unit 110 of the display device 100 determines whether a user operation related to the volume has been accepted. For example, the processing unit 110 determines whether a signal related to a volume change has been received from the remote controller 500 via the communication unit 130.

[0034] When a user operation related to the volume has been received (step S101: Yes), the processing unit 110 stores volume operation information indicating the user operation in the storage unit 120. The volume operation information is information that associates the operation content with the operation timing.

[0035] The operation content here is information that specifically indicates how the operation to change the volume was performed. For example, the operation content may include a pre-change volume that indicates the output volume before the user operation and a post-change volume that indicates the output volume after the user operation.

[0036] A user operation may be, for example, a collection of multiple operations of a volume change button. For example, if the volume up button is pressed once and then pressed again before a given time has elapsed, the multiple operations may be processed as a single unit. More specifically, for example, if the volume up button is a button for increasing the volume by one level and a user operation is performed in which the volume up button is pressed N times (N is an integer equal to or greater than 2) within a given time, the user operation may be recognized as one unit of operation that increases the volume by N levels. In this case, the pre-change volume is information that represents the level of the output volume before the volume up button is pressed, and the post-change volume is information that the output level has increased by N levels compared to the pre-change volume. However, the user operation may also be information in units corresponding to a single press of a button or the like.

[0037] FIG. 4A is an example of volume operation information stored in storage unit 120. The operation time in FIG. 4A corresponds to the operation timing described above. Here, information on the year, month, date, hour, and minute is exemplified as the operation timing, but the data format of the operation timing is not limited to this. As described above, the pre-change volume and post-change volume represent the output volume levels before and after a user operation. One row in the table in FIG. 4A is information representing one unit of user operation. By repeatedly executing the process shown in FIG. 3, information representing multiple user operations is accumulated as volume operation information.

[0038] If a user operation related to the volume has not been received (step S101: No), the processing unit 110 returns to step S101 and continues the process. In other words, the processing unit 110 waits without performing the processes from step S102 onwards until a user operation related to a volume change is received.

[0039] After recording the volume operation information, in step S103, the processing unit 110 determines whether the user's hearing has deteriorated based on the volume operation information for a predetermined period. The predetermined period here may be, for example, three months as described above. In this case, the processing unit 110 performs a process of reading out, as the processing target, data from the voice operation information stored in the storage unit 220, for which the difference between the operation timing (operation time) and the current time is three months or less. In this way, it is possible to determine whether the user's hearing has deteriorated during the period based on the voice operation information for the most recent three months. However, the end of the predetermined period is not limited to the current time, and another time may be used.

[0040] In the process of step S103, the processing unit 110 may determine whether the user's hearing has deteriorated based on volume operation information in which the operation timing belongs to a predetermined time period within a predetermined period. Here, the predetermined time period may represent a portion of each of the plurality of sub-periods, for example, when the predetermined period is divided into a plurality of sub-periods. For example, the sub-period may be one day, and the predetermined time period may represent a time period within the day. The time period may be one of the three periods when a day is divided into morning, afternoon, and night, or may be one hour out of the 24 hours into which a day is divided. Naturally, the length of the sub-period is not limited to one day. Furthermore, the length of the predetermined time period is not limited to the above example.

[0041] For example, even when the same user watches content using the same display device 100, the viewing environment, specifically, the state of surrounding sounds, changes depending on the time of day. The surrounding sounds may be sounds generated outdoors (such as car sounds or people talking) or sounds generated in nearby homes (such as sounds from the next room in an apartment building). If the user lives with family members, sounds generated by the housemates may be included in the surrounding sounds. Therefore, even if the user's hearing ability has not changed, the output volume level may be high when the surroundings are noisy and low when the surroundings are quiet. In this regard, by using voice operation information from the same time period for hearing ability assessment as described above, it is possible to suppress variations in the surrounding environment over time and accurately assess the user's hearing ability.

[0042] Furthermore, when a user lives with family members or the like, the same display device 100 may be shared by multiple users, including a first user and a second user. In this case, a situation may arise in which the first user has good hearing and a low output volume level is not a problem, while the second user has poor hearing and may increase the output volume level. In this case, the volume operation information may include information about both the first user and the second user, which may reduce the accuracy of the hearing assessment. In this regard, by using voice operation information from the same time period for the hearing assessment, it is possible to prevent information about multiple users from being mixed together. This is because each user has their own lifestyle habits, and it is considered that the same user is likely to watch the display device 100 at the same time period. Note that when considering the user's lifestyle habits, the predetermined time period is not limited to a time period within a day, but may be information specified by a combination of a day of the week and a time period.

[0043] 4B shows an example of volume operation information extracted from the volume operation information shown in FIG. 4A, in which the operation timing falls within a predetermined time period. The predetermined time period here may be "morning" or "around 10 o'clock."

[0044] Here, the processing unit 110 may determine whether the user's hearing has deteriorated based on at least one of the pre-change volume and the post-change volume. For example, the processing unit 110 determines whether the increase in volume over a predetermined period is equal to or greater than a predetermined value based on at least one of the pre-change volume and the post-change volume.

[0045] The processing unit 110 may determine whether the user's hearing ability has deteriorated by comparing multiple pre-change volume levels acquired at different times. For example, the processing unit 110 may determine that the user's hearing ability has deteriorated when the pre-change volume level acquired at a certain time level (in a narrow sense, the pre-change volume level of the voice operation information acquired last) acquired at a later time level is higher than the pre-change volume level acquired at that time level by a predetermined threshold or more.

[0046] The pre-change volume is information representing the volume level at which the user determines that the output volume is difficult to hear at the current volume level. For example, in the example of FIG. 4B, when considering the voice operation information acquired on October 12, it can be seen that the user found the output volume at level 7 difficult to hear and performed an operation to increase the volume level. In contrast, when considering the voice operation information acquired on January 1, it can be seen that the user found the output volume at level 10, which is relatively high, difficult to hear and performed an operation to increase the volume level. In this way, when the pre-change volume has been increasing over a predetermined period, it is considered that the user has begun to find it difficult to hear even loud sounds, and therefore the processing unit 110 determines that the user's hearing has deteriorated.

[0047] Alternatively, the processing unit 110 may determine whether the user's hearing ability has deteriorated by comparing multiple post-change volume levels acquired at different times. For example, the processing unit 110 may determine that the user's hearing ability has deteriorated when the post-change volume level acquired at a certain time level (in a narrow sense, the post-change volume level of the voice operation information acquired last) acquired at a later time level is higher than the post-change volume level acquired at that time level by a predetermined threshold or more.

[0048] The post-change volume is information representing a volume level at which the user is determined to be able to hear the sound sufficiently. For example, in the example of FIG. 4B, when considering the voice operation information acquired on October 12, it can be seen that the user was able to hear the sound without any problems by increasing the output volume level to 8. In contrast, when considering the voice operation information acquired on January 1, it can be seen that the user was finally able to hear the sound sufficiently by increasing the output volume level to a relatively high level of 11. In this way, when the post-change volume increases over a predetermined period, it is considered that the sound cannot be heard sufficiently unless the volume is increased, and therefore the processing unit 110 determines that the user's hearing ability has deteriorated. Note that the threshold used to determine the increase amount is, for example, about 3 levels, but the specific value can be varied in various ways.

[0049] Alternatively, the processing unit 110 may determine whether the user's hearing has deteriorated based on a comparison process between the pre-change volume at a first timing within a predetermined period and the post-change volume at a second timing within the predetermined period that is different from the first timing.

[0050] The second timing here may be a timing earlier than the first timing. For example, the processing unit 110 may determine the hearing loss of the user based on a comparison process between the post-change volume acquired at a certain timing and the pre-change volume acquired at a later timing (in a narrow sense, the pre-change volume of the voice operation information acquired last).

[0051] For example, in the example of FIG. 4B , when considering the post-change volume of the voice operation information acquired on October 12, it can be seen that the user could hear without any problems if the operation to increase the output volume level to 8 was performed. In contrast, when considering the pre-change volume of the voice operation information acquired on January 1, it can be seen that the user found it difficult to hear at an output volume of level 10 and performed an operation to increase the volume level. In this case, the user found it difficult to hear even at an output volume that had previously been judged to be sufficient for hearing or at a higher output volume level. In this way, when the pre-change volume at a relatively later timing is at the same level or has increased compared to the post-change volume at a relatively earlier timing, the processing unit 110 determines that the degree of increase in volume required by the user is high, and determines that the user's hearing ability has deteriorated.

[0052] The second timing may be later than the first timing. For example, the processing unit 110 may determine whether the user has a hearing impairment based on a comparison process between a pre-change volume acquired at a certain timing and a post-change volume acquired at a later timing (in a narrow sense, the post-change volume of the voice operation information acquired last). For example, the processing unit 110 may determine whether the user has a hearing impairment when the increase in the post-change volume at a relatively later timing is greater than or equal to a predetermined threshold value compared to the pre-change volume at a relatively earlier timing.

[0053] Furthermore, the specific process of determining hearing ability based on volume operation information for a predetermined period is not limited to the above example, and various modifications are possible.

[0054] Returning to Fig. 3, the description will continue. Through the above processing, the hearing assessment is performed in step S103. In step S104, the processing unit 110 determines whether the hearing assessment has determined that the user's hearing has deteriorated. If the hearing has deteriorated (step S104: Yes), in step S105, the processing unit 110 performs notification processing.

[0055] For example, when it is determined that the user's hearing ability has decreased, the processing unit 110 may perform processing to display a screen on the display unit 140 of the display device 100 that prompts the user to check their hearing ability. As described above, the processing unit 110 determines the hearing ability based on the user's operation regarding the output volume of the display device 100, and therefore, it is highly likely that the target user is viewing content using the display device 100. Therefore, by providing a notification using the display device 100, it is possible to appropriately inform the user that their hearing ability may be decreasing.

[0056] FIG. 5 is an example of a notification screen displayed in step S105. As shown in FIG. 5, display unit 140 may display a screen including text such as "The sound is louder than before. We recommend that you check your hearing." Note that FIG. 5 shows an example in which the text is displayed in an area near the center of display unit 140, but this is not limiting. For example, processing unit 110 may display the above text in a superimposed manner in a peripheral area of ​​display unit 140 while displaying video data corresponding to the content on display unit 140. In addition, various modifications of the screen used for notification are possible.

[0057] In the above description, an example has been described in which the process of acquiring and storing volume operation information (steps S101-S102) and the hearing assessment and notification process (steps S103-S105) are executed consecutively. In other words, an example has been described in which the acquisition of voice operation information is used as a trigger to execute the hearing assessment and notification process. In this way, it becomes possible to execute the hearing assessment every time the latest voice operation information is acquired (for example, every time an operation of the remote controller 500 is accepted).

[0058] However, the method of the present embodiment is not limited to this, and the process of acquiring and storing volume operation information (steps S101-S102) and the hearing assessment and notification process (steps S103-S105) may be performed independently. For example, recording of volume operation information is performed each time the remote controller 500 is operated. The hearing assessment may be performed periodically, such as once a day, when the TV is turned on, or based on another trigger. For example, as will be described later with reference to FIG. 6, when the terminal device 200 (or, in a narrow sense, the terminal device 200-2 used by the user's family) and the display device 100 cooperate with each other, the hearing assessment may be triggered by a request from the terminal device 200. In this way, it is possible to flexibly set the timing of the hearing assessment while appropriately accumulating voice operation information.

[0059] 3. Variations Some modified examples will be described below.

[0060] 3.1 What to do if you are diagnosed with hearing loss The above describes an example in which, when it is determined that hearing loss has occurred, notification processing is performed using the display unit 140 of the display device 100 (FIG. 5). However, the method of the present disclosure is not limited to this. For example, the notification processing may be performed in the terminal device 200 associated with the display device 100.

[0061] FIG. 6 is a diagram illustrating a configuration example of an information processing system 10 including a terminal device 200. As illustrated in FIG. 6, the information processing system 10 may include the terminal device 200 and a server system 300 in addition to the audio output device 20 (display device 100) and the remote controller 500. The terminal device 200 here may be a terminal device 200-1 used by a user of the display device 100, or a terminal device 200-2 connected to the display device 100 via a network, or both. The terminal device 200-2 is a device used by, for example, a family member who lives separately from the user of the display device 100. The display device 100, the terminal device 200-2, and the server system 300 are connected via a network. Furthermore, the terminal device 200-1 may be connected to other devices, such as the display device 100 and the server system 300, via a network.

[0062] Fig. 7 is a block diagram showing a detailed configuration example of the terminal device 200. The terminal device 200 may include, for example, a processing unit 210, a storage unit 220, a communication unit 230, a display unit 240, an operation unit 250, and an imaging unit 260. These components are connected to each other via, for example, a bus provided in the terminal device 200. However, the configuration of the terminal device 200 is not limited to that shown in Fig. 7, and modifications such as omitting some components or adding other components are possible. The terminal device 200-1 and the terminal device 200-2 each have, for example, the configuration shown in Fig. 7. However, the configuration of the terminal device 200-1 and the terminal device 200-2 may differ.

[0063] The processing unit 210 is configured by hardware including at least one of a circuit for processing digital signals and a circuit for processing analog signals. The processing unit 210 may also be realized by a processor. Various types of processors, such as a CPU, a GPU, or a DSP, can be used as the processor. The processor executes instructions stored in the memory of the terminal device 200, thereby realizing the functions of the processing unit 210 as processing.

[0064] The storage unit 220 is a work area for the processing unit 210, and is realized by various types of memory such as SRAM, DRAM, and ROM.

[0065] The communication unit 230 is an interface for performing communication via a network, and includes, for example, an antenna, an RF circuit, and a baseband circuit. The communication unit 230 performs communication with the display device 100.

[0066] For example, the communication unit 230 of the terminal device 200-1 is connected to the display device 100 by wireless communication conforming to the IEEE802.11 or Bluetooth standard. The communication unit 230 of the terminal device 200-2 is connected to the display device 100 by wireless communication conforming to the IEEE802.11 standard. The communication unit 230 may be connected to the display device 100 via a communication device such as a router.

[0067] The display unit 240 is an interface that displays various information and may be a liquid crystal display, an organic EL display, or another type of display. The operation unit 250 is an interface that accepts user operations. The operation unit 250 may be buttons or the like provided on the terminal device 200. The display unit 240 and the operation unit 250 may also be a touch panel that is integrally configured.

[0068] The imaging unit 260 has an image sensor that captures an image of a predetermined imaging range and outputs image information. The image information here may be a still image or a moving image. The image information may also be color or monochrome.

[0069] The terminal device 200 may also include components not shown in Fig. 7. For example, the terminal device 200 may have various sensors, such as motion sensors such as acceleration sensors and gyro sensors, pressure sensors, and GPS (Global Positioning System) sensors. The terminal device 200 may also include a light-emitting unit, a vibration unit, a sound input unit, a sound output unit, and the like. The light-emitting unit is, for example, an LED (light emitting diode) and provides notification by emitting light. The vibration unit is, for example, a motor and provides notification by vibration. The sound input unit is, for example, a microphone. The sound output unit is, for example, a speaker and provides notification by sound.

[0070] The server system 300 is a device having a processing unit, a storage unit, and a communication unit. The communication unit is connected to the display device 100 and the terminal device 200 via communication compliant with standards such as IEEE802.11 or IEEE802.3.

[0071] The server system 300 is, for example, a server that provides the service according to this embodiment, and is connected to the display device 100 and a terminal device 200 on which an application (for example, a remote control application) for using the display device 100 is installed, and collects information.

[0072] The processing executed by the display device 100 is similar to the example described above with reference to Fig. 3. That is, the processing unit 110 of the display device 100 acquires voice operation information based on a user operation (steps S101-S102), determines whether the user's hearing has deteriorated based on the voice operation information (step S103), and if it is determined that the user's hearing has deteriorated (step S104: Yes), executes notification processing (step S105).

[0073] 2, the display device 100 may include the communication unit 130. When it is determined that the user's hearing has deteriorated, the processing unit 110 may cause the terminal device 200 to execute a notification process, via the communication unit 130, to prompt the user to check their hearing. The notification here may be a display on a screen, or may be a notification using sound, light, vibration, or the like.

[0074] In this way, it becomes possible to execute notification processing using the terminal device 200. For example, in the case of the terminal device 200-1 of the user of the display device 100, it is assumed that the terminal device 200-1 is carried by the user. Therefore, it becomes possible to make the user more aware of notifications regarding hearing. For example, since the terminal device 200 is considered to be closer to the user than the display device 100, it is easy to make the user aware of the notification content even when a notification using sound is given to a user with hearing loss. Furthermore, since it is not necessary to use the display unit 140 of the display device 100, even if an image similar to that shown in FIG. 5 is displayed on the display unit 240 of the terminal device 200, the display of content such as a television program is not obstructed.

[0075] Alternatively, when the terminal device 200-2 of a user other than the user of the display device 100 (hereinafter referred to as the second user) is targeted, it is possible to notify the second user of the hearing loss of the user of the display device 100. For example, if the user of the display device 100 is elderly, even if the screen shown in FIG. 5 is displayed on the display unit 140 of the display device 100, it is conceivable that the elderly user will ignore the display itself, or will understand the displayed content but be unable to take appropriate action. In this regard, by targeting the terminal device 200-2 of the second user, such as a family member, as the target of the notification, it is possible to encourage the second user to provide support to the user of the display device 100. The support here may be suppressing noise by limiting the output volume of the display device 100, or may be accompanying the user to a medical examination or the fitting of a hearing aid at a hospital.

[0076] Here, the terminal device 200 to be notified may be the terminal device 200 associated with the display device 100. The association between the display device 100 and the terminal device 200 may be performed using, for example, wireless communication. For example, when the display device 100 and the terminal device 200 are connected using short-range wireless communication such as Bluetooth, the display device 100 and the terminal device 200 are associated with each other. Alternatively, the display device 100 and the terminal device 200 connected to the same network such as Wi-Fi may be associated with each other. The same network may be a network formed by one communication device (router) or a network identified by the same SSID (Service Set Identifier). Furthermore, the association between the display device 100 and the terminal device 200 may be performed manually by a user (including a second user).

[0077] Information associating the display device 100 with the terminal device 200 may be stored, for example, in a storage unit of the server system 300. In this way, it becomes possible to appropriately manage the correspondence relationship of the display device 100 even for a terminal device 200, such as the terminal device 200-2, that is used at a location distant from the display device 100. However, the information associating the display device 100 with the terminal device 200 may be stored in the storage unit 120 of the display device 100 or the storage unit 220 of the terminal device 200.

[0078] Furthermore, although an example of executing a notification urging the user to check their hearing has been described above as the notification process, the method of the present disclosure is not limited to this.

[0079] For example, when it is determined that the user has hearing loss, the processing unit 110 may perform a notification process to prompt the user to change the directionality or locality of the audio data output from the sound output unit. In this way, it is possible to prompt the user to change the output mode of the audio data to a mode suitable for the hearing loss.

[0080] Fig. 8 is a diagram showing an example of the configuration of the sound output unit of the display device 100. As described above with reference to Fig. 2, the sound output unit according to this embodiment may include a plurality of sound output units including a first sound output unit 160 and a second sound output unit 400. The first sound output unit 160 is, for example, a built-in speaker, and the second sound output unit is, for example, an external speaker.

[0081] As shown in Fig. 8, the multiple sound output units may have different directivities of audio data (specifically, directivities of audio signals that are output signals corresponding to audio data). In the example of Fig. 8, the first sound output unit 160, which is a built-in speaker, is a speaker with relatively low directivity, and radiates audio signals over a wide range. On the other hand, the second sound output unit 400, which is an external speaker, is a speaker with relatively high directivity, and the strength of the audio signal is high in a specific direction.

[0082] As described above, a user with hearing loss needs to increase the output volume level in order to hear the sound. As a result, a loud sound may be transmitted to the surrounding area, causing noise. In this regard, by using a directional speaker that increases the strength of the sound signal in front of the display device 100 and decreases the strength of the sound signal in other areas, it is thought that it is possible to deliver sound of sufficient strength to a user with hearing loss while suppressing noise in the surrounding area.

[0083] Therefore, when it is determined that the user's hearing has deteriorated, the processing unit 110 may execute a notification process to encourage the user to increase the directivity of the audio data on the display unit 140 or the terminal device 200. For example, when the sound output unit includes both a speaker with high directivity and a speaker with low directivity as described above, the processing unit 110 executes a notification process to encourage the user to use the speaker with high directivity.

[0084] 9 shows an example of a screen displayed on the display unit 140 of the display device 100 in this case. Here, an example is considered in which the directivity of the connected audio (external speaker, second sound output unit 400) is relatively high, so the display unit 140 displays a screen including text such as "The sound is louder than before. Would you like to switch to the connected audio?" In this way, if the user's hearing deteriorates, a speaker with high directivity is used, making it possible to suppress noise generation.

[0085] Alternatively, the second sound output unit 400 may be a wearable speaker worn by a user, or a portable speaker that can be carried around. The second sound output unit 400 is connected to the display device 100 by short-range wireless communication such as Bluetooth. The wearable speaker is, for example, a neckband speaker, but other types of speakers may also be used. The portable speaker may have any shape.

[0086] These speakers are smaller than the built-in speaker (first sound output unit 160) and are expected to be placed close to the user's ears during use. Therefore, these speakers are highly localized speakers that supply audio data to a localized area including the user. By using speakers with high localization, it is thought that it is possible to deliver audio data of sufficient intensity to a user with hearing loss while suppressing noise in the surrounding area.

[0087] Therefore, when it is determined that the user's hearing is impaired, the processing unit 110 may cause the display unit 140 or the terminal device 200 to execute a notification process that prompts the user to increase the locality of the audio data. For example, as described above, when the sound output unit includes both a speaker with high locality and a speaker with low locality, the processing unit 110 executes a notification process that prompts the user to use the speaker with high locality. Specifically, as described above, it is expected that an external speaker has higher locality than a built-in speaker. Therefore, the processing unit 110 may display a screen including text that prompts the user to change to connected audio (external speaker, second sound output unit 400), as in the example of FIG. 9.

[0088] Although the above describes an example of notification processing that prompts a change in the directionality or locality of audio data, the present invention is not limited to this. For example, the processing unit 110 may automatically change the directionality or locality when it is determined that the user's hearing has deteriorated. Specifically, when it is determined that the user's hearing has deteriorated, the processing unit 110 automatically changes the sound output unit to which the audio data is output.

[0089] 3.2 Examples of applications other than audio output devices (display devices) The above description has been given of an example in which the processing of each step shown in Fig. 3 is performed by the processing unit 110 of the audio output device 20 (display device 100). However, the method of the present disclosure is not limited to this.

[0090] For example, the technique of the present disclosure may be applied to the terminal device 200 shown in Fig. 6. The terminal device 200 here is, for example, the terminal device 200-1 used by the user of the display device 100, and in a narrow sense may be the terminal device 200 installed with a remote control application that has the function of the remote controller 500. In this case, the remote controller 500 may be omitted.

[0091] As shown in FIG. 7 , the terminal device 200 includes a communication unit 230, an operation unit 250, a processing unit 210, and a storage unit 220. The communication unit 230 communicates with the display device 100. The operation unit 250 accepts a user operation related to the output volume of audio data on the display device 100. The processing unit 210 performs processing to transmit a control signal for the output volume to the display device 100 via the communication unit 230 based on the user operation. The functions of the operation unit 250 and the processing unit 210 are realized, for example, by a processor of the terminal device 200 operating in accordance with a remote control application. The storage unit 220 stores volume operation information in which the operation content and operation timing of the user operation are associated. The processing unit 210 then determines whether the user's hearing has deteriorated based on the volume operation information acquired over a predetermined period.

[0092] In this way, it becomes possible to execute the processing of this embodiment shown in Fig. 3 in the terminal device 200. When it is determined that the user's hearing has deteriorated, the processing unit 210 of the terminal device 200 may execute notification processing in the terminal device 200 using the display unit 240 or the like, or may cause the display unit 140 of the display device 100 to execute notification processing. The content of the notification may be a prompt to check the hearing (Fig. 5), or a prompt to change the directionality or locality of the audio data (Fig. 9).

[0093] The technique of the present disclosure may also be applied to an information processing system 10 including a server system 300 shown in Fig. 6. The information processing system 10 includes an audio output device 20 and the server system 300. The audio output device 20 has a processing unit 110 that controls the output of audio data. The server system 300 communicates with the audio output device 20. When the audio output device 20 receives a user operation related to the output volume of audio data, the server system 300 acquires volume operation information that associates the operation content and operation timing, and determines whether the user has a hearing impairment based on the volume operation information acquired over a predetermined period.

[0094] 3 can be executed in the server system 300. In this case, even if the terminal device 200-2 to be notified is located at a distance from the display device 100, for example, the notification process can be executed smoothly.

[0095] Furthermore, the processing according to the present disclosure is not limited to being executed in any one of the audio output device 20, the terminal device 200, and the server system 300. For example, the processing shown in Fig. 3 may be realized by distributed processing of two or more of the audio output device 20, the terminal device 200, and the server system 300. For example, the processing of steps S101 and S103-S105 in Fig. 3 may be executed by the processing unit 110 of the audio output device 20, and the storage of the voice operation information in step S102 may be executed by the storage unit of the server system 300.

[0096] 3.3 Example of judgment based on volume operation information 4A and 4B, an example has been described in which it is determined whether the user's hearing has deteriorated based on changes in volume operation information over a predetermined period (e.g., three months). Specifically, the processing unit 110 compares the pre-change volume or post-change volume acquired at a first timing with the pre-change volume or post-change volume acquired at a second timing different from the first timing. However, the method of the present disclosure is not limited to this.

[0097] For example, the processing unit 110 of the display device 100 may acquire, as the operation content of the volume operation information at a given timing, a pre-change volume representing the output volume before the user operation and a post-change volume representing the output volume after the user operation. The processing unit 110 then determines that the user's hearing ability has deteriorated if the increase in the post-change volume relative to the pre-change volume is equal to or greater than a first threshold. In this way, it becomes possible to determine the user's hearing ability based on the user operation at a given timing.

[0098] In a narrow sense, the given timing here refers to the timing when the most recent user operation was performed, and the user's hearing assessment may be triggered by the acquisition of voice operation information at that timing. In this way, when an extreme operation is performed, it can be quickly determined that the user's hearing has deteriorated and a notification process can be executed. As a result, it becomes possible to quickly suppress noise, etc.

[0099] 10 is an example of volume operation information at a given timing. In this example, the pre-change volume level is 10, and the post-change volume level is 20. Therefore, the increase in post-change volume relative to the pre-change volume is 10. The first threshold here is a value greater than the threshold (e.g., 3 levels) used in determinations over a predetermined period, and may be, for example, around 5 levels. In this way, if a single user operation causes an extreme change in volume, it can be assumed that there is some kind of hearing problem, regardless of long-term changes.

[0100] Furthermore, processing unit 110 determines that the user's hearing has deteriorated if the changed volume is equal to or greater than a second threshold. For example, the second threshold corresponds to a level of output volume that is loud enough to cause noise in the surrounding area, and may be determined based on, for example, the model number of display device 100 or the characteristics of built-in first sound output unit 160. In this case, too, if the volume is not loud enough to cause noise, the user will have difficulty hearing, so it is possible to determine the user's hearing has deteriorated from volume operation information at one timing.

[0101] The determination process based on voice operation information at one timing may be combined with, for example, the notification process for prompting a change in the directionality or localization of the audio data. As described above, increasing the directivity or localization can provide the user with sufficiently loud audio data while suppressing surrounding noise. When a user operation that involves a drastic change in volume or a user operation that excessively increases the volume after the change is performed, it is highly likely that the user operation has already generated noise, so a notification for prompting a change in the audio output unit, which can directly suppress the noise, is effective. For example, the processing unit 110 may perform the notification process for prompting a hearing assessment shown in FIG. 5 as the notification process based on voice operation information for a predetermined period, and the notification process for prompting a change in the audio output unit shown in FIG. 9 as the notification process based on voice operation information at one timing. However, the determination process based on voice operation information at one timing may be combined with the notification process for prompting a hearing check shown in FIG. 5.

[0102] Furthermore, in the method of the present disclosure, processing based on volume operation information at a given timing may be performed, and processing based on voice operation information for a predetermined period may be omitted. For example, the method of the present disclosure may be applied to an audio output device 20 that includes a processing unit 110 that controls the output of audio data and a storage unit 120 that, when a user operation related to the output volume of audio data is received, stores volume operation information indicating the content of the user operation, and the processing unit 110 determines whether the user has hearing loss based on the volume operation information. As with the above-mentioned examples, the method of the present disclosure may be applied to a terminal device 200, a server system 300, and an information processing system 10 that have similar configurations.

[0103] Although the present embodiment has been described in detail above, those skilled in the art will readily understand that many modifications are possible without substantially departing from the novel features and advantages of the present embodiment. Therefore, all such modifications are intended to be included within the scope of the present disclosure. For example, a term described at least once in the specification or drawings with a different term having a broader or equivalent meaning may be replaced with that different term anywhere in the specification or drawings. Furthermore, all combinations of the present embodiment and modifications are also intended to be included within the scope of the present disclosure. Furthermore, the configurations and operations of audio output devices, display devices, terminal devices, server systems, information processing systems, etc. are not limited to those described in the present embodiment, and various modifications are possible. [Explanation of symbols]

[0104] 10 Information processing system, 20 Audio output device, 100 Display device, 110 Processing unit, 120 Storage unit, 130 Communication unit, 140 Display unit, 150 Operation unit, 160 First audio output unit, 200, 200-1, 200-2 Terminal device, 210 Processing unit, 220 Storage unit, 230 Communication unit, 240 Display unit, 250 Operation unit, 260 Imaging unit, 300 Server system, 400 Second audio output unit, 500 Remote controller

Claims

1. a processing unit that controls the output of audio data; a storage unit configured to store volume operation information in which the operation content and operation timing of the user operation are associated with each other when the user operation related to the output volume of the audio data is received; Including, The processing unit An audio output device that determines whether or not a user's hearing has deteriorated based on the volume operation information acquired over a predetermined period.

2. In claim 1, The processing unit The audio output device determines whether the hearing ability of the user is impaired based on the volume operation information, the operation timing of which belongs to a predetermined time period within the predetermined period.

3. In claim 2, The processing unit acquiring, as the operation content of the volume operation information, at least one of a pre-change volume representing the output volume before the user operation and a post-change volume representing the output volume after the user operation; An audio output device that determines whether the user's hearing ability has deteriorated based on at least one of the pre-change volume and the post-change volume.

4. In claim 3, The processing unit An audio output device that determines whether the user's hearing has deteriorated based on a comparison process between the pre-change volume at a first timing within the specified period and the post-change volume at a second timing within the specified period that is different from the first timing.

5. In any one of claims 1 to 4, The processing unit An audio output device that performs a notification process to prompt a user to change the directionality or locality of the audio data output from a sound output unit when it is determined that the user's hearing has deteriorated.

6. In any one of claims 1 to 4, Further including a display unit for displaying video data, The processing unit An audio output device that, when it is determined that the user's hearing has deteriorated, performs processing to display on the display unit a screen that prompts the user to check their hearing.

7. In any one of claims 1 to 4, a communication unit that communicates with a terminal device associated with the audio output device, The processing unit An audio output device that, when it is determined that the user's hearing has deteriorated, causes the terminal device to execute a notification process via the communication unit to prompt the user to check their hearing.

8. In claim 1 or 2, acquiring, as the operation content of the volume operation information at a given timing, a pre-change volume representing the output volume before the user operation and a post-change volume representing the output volume after the user operation; An audio output device that determines that the user's hearing is impaired if the increase in the post-change volume relative to the pre-change volume is equal to or greater than a first threshold, or if the post-change volume is equal to or greater than a second threshold.

9. a communication unit that communicates with an audio output device; an operation unit that accepts a user operation regarding an output volume of audio data in the audio output device; a processing unit that performs processing to transmit a control signal for the output volume to the audio output device via the communication unit based on the user operation; a storage unit that stores volume operation information in which the operation content and operation timing of the user operation are associated with each other; Including, The processing unit The terminal device determines whether the user's hearing ability has deteriorated based on the volume operation information acquired over a predetermined period of time.

10. Accepting a user operation regarding the output volume of audio data in the audio output device; Acquire volume operation information in which the operation content and operation timing of the user operation are associated with each other; determining whether the user has hearing loss based on the volume operation information acquired over a predetermined period of time; Information processing methods.

Images