Information processing device, television receiver, display control system, information processing method, information processing program, and terminal device
The information processing apparatus dynamically adjusts television output modes based on user reactions, addressing the cumbersome manual adjustment issue by automatically aligning settings with user preferences.
Patent Information
- Application Number
- PCT/JP2026/018119
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2026-05-13
- Publication Date
- 2026-11-19
AI Technical Summary
Existing systems require users to manually adjust output modes of devices like television receivers, which can be cumbersome when the automatically determined mode is not preferred.
An information processing apparatus that acquires user reactions to notification information and adjusts the output mode of devices like televisions based on this reaction information, using components such as input units, receiving units, and control units to dynamically modify image and audio settings.
Enables easy adjustment of the output mode to the user's preference, eliminating the need for manual settings and enhancing user satisfaction.
Smart Images

Figure JP2026018119_19112026_PF_FP_ABST
Abstract
Description
Information Processing Apparatus, Television Receiver, Display Control System, Information Processing Method, Information Processing Program, and Terminal Device
[0001] The present invention relates to an information processing apparatus, a television receiver, a display control system, an information processing method, an information processing program, and a terminal device.
[0002] In techniques for modifying the output mode of an output device that outputs images and audio, such as a television receiver, configurations are known that execute processing based on user reactions. For example, Patent Document 1 describes a television receiver that determines operations such as image mode switching based on sensing results. The television receiver presents the reason for the executed determined operation based on an estimation result for the user's reaction.
[0003] International Publication No. 2021 / 205742
[0004] However, in the television receiver described in Patent Document 1, even if the presented operation reason is reasonable for the user, the output mode after the determined operation is executed is not necessarily the user's preferred output mode. In addition, if the output mode after the determined operation is executed is not the user's preferred output mode, the user needs to change the settings manually. In this case, the user is required to perform detailed settings, which feels cumbersome.
[0005] An aspect of the present invention aims to realize a technique that can easily adjust the output mode of an output device to the output mode desired by the user.
[0006] To solve the above problem, an information processing apparatus according to an aspect of the present invention includes a first acquisition unit that acquires reaction information indicating a user's reaction to notification information notified to the user from an output device, and a first control unit that changes the output mode of the output device based on the notification information and the reaction information.
[0007] To solve the above problems, a display control system according to one aspect of the present invention includes a receiving device for receiving content data relating to content, an output device for outputting the content, an acquisition device for acquiring reaction information indicating the user's response to notification information notified to the user by the output device, and an output mode changing device for changing the output mode of the content by the output device based on the notification information and the reaction information.
[0008] To solve the above problems, an information processing method according to one aspect of the present invention includes a first acquisition step of acquiring reaction information indicating the user's response to notification information notified to the user from an output device, and a control step of changing the output mode of the output device based on the notification information and the reaction information.
[0009] To solve the above problems, an information processing program according to one aspect of the present invention causes a computer to perform a first acquisition process to acquire reaction information indicating the user's response to notification information notified to the user from an output device, and a control process to change the output mode of the output device based on the notification information and the reaction information.
[0010] According to one aspect of the present invention, the output mode of the output device can be easily adjusted to the output mode desired by the user.
[0011] This is a block diagram showing the configuration of a display device according to Embodiment 1 of the present invention. This is a diagram showing an example of using a display device according to Embodiment 1 of the present invention. This is a diagram showing an example of displaying a display image according to Embodiment 1 of the present invention. This is a functional block diagram showing a specific configuration example of an information processing device according to Embodiment 1 of the present invention. This is a flowchart showing an example of the processing flow by an information processing device according to Embodiment 1 of the present invention. This is a flowchart showing a specific processing flow by an information processing device according to Embodiment 1 of the present invention. This is a diagram showing an example of an image displayed on the display unit according to Embodiment 1 of the present invention. This is a flowchart showing another example of the processing flow by an information processing device according to Embodiment 1 of the present invention. This is a flowchart showing yet another example of the processing flow by an information processing device according to Embodiment 1 of the present invention. This is a flowchart showing yet another example of the processing flow by an information processing device according to Embodiment 1 of the present invention. This is a diagram showing yet another example of the processing flow by an information processing device according to Embodiment 1 of the present invention. This is a diagram showing another example of an image displayed on the display unit according to Embodiment 1 of the present invention. This is a flowchart showing yet another example of the processing flow by an information processing device according to Embodiment 1 of the present invention. This is a diagram showing yet another example of an image displayed on the display unit according to Embodiment 1 of the present invention.
[0012] [Embodiment 1] An embodiment of the present invention will be described in detail below.
[0013] <Display Device 1> The display device 1 according to this embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the display device 1 according to this embodiment. As shown in Figure 1, the display device 1 includes an input unit 10, a receiving unit 20, an image quality control unit 30, an sound quality control unit 40, a synthesis unit 50, a display unit 60 (output device), a speaker 70 (output device), and an information processing device 100.
[0014] It should be noted that the parts shown in Figure 1 are merely examples of the configuration of the display device 1, and the configuration of the display device 1 is not limited to this configuration. For example, the display device 1 may include various components such as a remote control device operated by the user, or a storage unit for storing images.
[0015] Furthermore, the components of the display device 1 may be distributed across multiple devices via a network. In such cases, the display device 1 may also be referred to as a display control system.
[0016] Furthermore, the information processing device 100 may be configured to include at least one of the following in the display device 1: an input unit 10, a receiving unit 20, an image quality control unit 30, an sound quality control unit 40, and a synthesis unit 50. Alternatively, the information processing device 100 may be configured to include at least one of the following in the display device 1: a display unit 60 and a speaker 70.
[0017] Furthermore, although the example shown in Figure 1 includes an information processing device 100, this embodiment is not limited to this, and the information processing device 100 may be implemented as a set-top box connected to the display device 1.
[0018] (Input Unit 10) The input unit 10 acquires input information input to the display device 1 and supplies the acquired input information to the information processing device 100. In other words, the input unit 10 can also be said to be an acquisition device that acquires input information. Examples of the input unit 10 include a sensor that acquires sensing data sensing a user using the display device 1 as input information, a sensor that acquires sensing data sensing the surroundings of the display device 1 as input information, and an operation signal receiving unit that acquires operation signals from a remote control device as input information. The input unit 10 may also be a communication interface that acquires input information from other devices. For example, the input unit 10 may be connected to an external device equipped with the sensing function described above and configured to communicate with the external device. In this case, the input unit 10 may acquire sensing data from the external device. As an example, the input unit 10 acquires audio data collected by a microphone connected via a USB (Universal Serial Bus) interface.
[0019] Examples of sensing data that senses a user using the display device 1 include audio data representing the sound collected by one or more microphones that the display device 1 is equipped with to collect the user's speech. For example, operation information indicating the user's voice operation can be considered as an example of sensing data. Another example of sensing data that senses a user using the display device 1 is imaging data representing the user captured by one or more cameras that the display device 1 is equipped with. Furthermore, an example of sensing data that senses the area around the display device 1 is illuminance information indicating the brightness around the display device 1. Additionally, a reflective object detection device such as LiDAR (Laser Imaging Detection and Ranging) may be used as the sensing device. Furthermore, the display device 1 may be equipped with a sensor using infrared, millimeter wave, or ultrasonic waves, and the data acquired by this sensor may be used as sensing data.
[0020] (Receiving Unit 20) The receiving unit 20 receives content data and supplies the received content data to the information processing device 100. In other words, the receiving unit 20 can also be described as a receiving device that receives content data related to content. Examples of content data received by the receiving unit 20 include encoded images, encoded audio data, and related information associated with the images. Examples of encoded images include data encoded using various video encoding technologies such as MPEG2, MPEG4, H.264, and H.265 (e.g., TS (Transport Stream)).
[0021] Furthermore, examples of related information include image format information (information indicating whether it is an HDR (High Dynamic Range) signal (information indicating whether it is SDR (Standard Dynamic Range), HLG (Hybrid Log Gamma), or PQ (Perceptual Quantization) (HDR10))), image color space information (information indicating whether it is BT (Broadcasting Television) .709 or BT.2020), program information related to the content, program guide data including program information, data broadcasting information provided in relation to the content, and descriptive information related to the content. In addition, related information may be information that can be extracted from the content data without decoding processing using video encoding technologies such as MPEG2 as described above.
[0022] Furthermore, while the following explanation primarily focuses on video content, the content decrypted from the content data may also include still images, audio, text, or web pages. For example, video content may include any images related to entertainment, education, news, e-commerce, advertising, accommodation information, travel arrangements, etc.
[0023] Furthermore, the receiving unit 20 may be configured to acquire the content data from the Internet via wireless or wired communication. Alternatively, the receiving unit 20 may be configured to acquire the content data from broadcast waves and may be equipped with a tuner for selecting one of the multiple channels included in the broadcast waves. When the receiving unit 20 is equipped with a tuner, the display device 1 is also referred to as a television receiver (or simply a television).
[0024] If the content obtained by decrypting content data includes image and audio data, the image will also be referred to as image content, and the audio data will also be referred to as audio content.
[0025] (Image Quality Control Unit 30) The image quality control unit 30 modifies the image quality of the image content output from the information processing device 100 by applying image quality parameters output from the information processing device 100. Examples of image quality parameters include parameters indicating the degree of brightness, contrast, color intensity, and sharpness, as well as image quality modes corresponding to each of multiple categories. The image quality control unit 30 supplies the image content to which the image quality parameters have been applied to the synthesis unit 50.
[0026] Alternatively, instead of or in addition to the configuration shown in Figure 1, the image quality control unit 30 may be configured to acquire notification information, which will be described later, from the information processing device 100. In this case, the image quality control unit 30 changes the image quality of the image included in the notification information (hereinafter also referred to as the notification image) by applying image quality parameters output from the information processing device 100. The image quality control unit 30 supplies the notification image to which the image quality parameters have been applied to the synthesis unit 50.
[0027] Notification information refers to information that is communicated to the user using the display device 1. The notification can be, for example, visual, auditory, or through at least one of the other senses. As an example, the notification information may include at least one of a predetermined image and audio data including speech mediated by the predetermined image. In this case, the display device 1 communicates the notification content to the user in at least one of the following forms: display via the predetermined image and / or speech. The predetermined image may also include a character object, a background image of the character, or a computer-generated image obtained from an external source by the input unit 10.
[0028] Furthermore, if the notification information includes a predetermined image and audio data including speech mediated by the predetermined image, and the predetermined image is a character, the character may be referred to as an avatar. In this case, the display device 1 may output the audio data included in the notification information while displaying it in a way that makes it appear as if the avatar is speaking, by changing the avatar's facial expressions and mouth shape. Also, as described above, the display device 1 can acquire audio data indicating the user's response to the audio data spoken by the avatar via the microphone, allowing the user to converse with the avatar. The term "avatar" is not limited to a representation of the user themselves in the general sense. For example, the avatar may be a character selected by the user from among multiple characters.
[0029] The notification information may be output in response to user input, or it may be output at any frequency regardless of user input. Examples of configurations where the information is output in response to user input include a configuration where the notification information is output when input information indicating that the notification information should be output is received from the remote control device, and a configuration where the notification information is output when input information including voice data indicating user speech is received from the microphone. Examples of arbitrary frequencies include a configuration where the notification information is output at predetermined intervals (every 10 minutes, every hour, etc.) and a configuration where the notification information is output when the content changes.
[0030] (Sound Quality Control Unit 40) The sound quality control unit 40 modifies the sound quality of the audio content output from the information processing device 100 by applying audio parameters output from the information processing device 100. Examples of sound quality parameters include parameters indicating frequency characteristics and sound quality modes corresponding to each of multiple categories. The sound quality control unit 40 supplies the audio content to which the sound quality parameters have been applied to the synthesis unit 50.
[0031] Alternatively, instead of or in addition to the configuration shown in Figure 1, the sound quality control unit 40 may be configured to acquire notification information from the information processing device 100. In this case, the sound quality control unit 40 modifies the sound quality of the audio data (hereinafter also referred to as notification audio) included in the notification information by applying sound quality parameters output from the information processing device 100. The sound quality control unit 40 supplies the notification audio to which the sound quality parameters have been applied to the synthesis unit 50.
[0032] (Combination Unit 50) The combination unit 50 combines the image content supplied from the image quality control unit 30, the audio content supplied from the sound quality control unit 40, and the notification information supplied from the information processing device 100.
[0033] More specifically, the synthesis unit 50 generates a display image by combining the image content supplied from the image quality control unit 30 with the image included in the notification information supplied from the information processing device 100. The synthesis unit 50 then supplies the generated display image to the display unit 60.
[0034] Furthermore, the synthesis unit 50 generates output audio by combining the audio content supplied from the sound quality control unit 40 with the audio data included in the notification information supplied from the information processing device 100. The synthesis unit 50 supplies the generated output audio to the speaker 70.
[0035] The above example is not limited to this embodiment, and as an example, the display image may be configured to include either a content image or a notification image but not the other. More specifically, the display image may consist of either a content image or a notification image.
[0036] Similarly, the output audio may be configured to include either audio content or notification audio, but not the other. More specifically, the output audio may consist of, for example, audio content, or notification audio.
[0037] (Display Unit 60) The display unit 60 displays the display image generated by the synthesis unit 50. The display image includes at least one of a still image and a moving image (video). As mentioned above, the display image includes, for example, at least one of a content image and informational information. In other words, the display unit 60 displays at least one of a content image and informational information. Therefore, the display device 1 and the display unit 60 can also be said to be output devices that output images.
[0038] Examples of the display unit 60 include a display panel and a driver that drives the display panel based on image data of the display image. Examples of the display panel include, but are not limited to, liquid crystal panels, organic EL panels, or plasma display panels; the term "display panel" refers to any panel that displays images. Alternatively, the display unit 60 includes any surface capable of displaying images, such as an indoor or outdoor wall, or a part of the body, such as an arm. Examples of displays shown on the display unit 60 will be described later.
[0039] (Speaker 70) Speaker 70 outputs output sound generated by the synthesis unit 50 to the user. As described above, this output sound includes, for example, at least one of audio content and notification sound. In other words, at least one of audio content and notification sound is output from speaker 70. Therefore, it can be said that the display device 1 and speaker 70 are output devices that output sound. In this embodiment, the term "sound" may include, but is not limited to, human voices, and refers to sound in general that propagates through a medium such as air.
[0040] (Example of use of display device 1) Figure 2 shows an example of use of the display device 1. In the example shown in Figure 2, the display device 1 is implemented as a stationary display device equipped with legs. As shown in Figure 2, the display device 1 is configured to include an input unit 10 for sensing the user U or the surroundings of the display device 1, a display unit 60 for displaying an image, and speakers 70 (two in the example of Figure 2) for outputting audio. The display device 1 may also be implemented as a wall-mounted display device. Furthermore, the display device 1 is not limited to a television, for example, but may be a projection device such as a monitor or projector that does not have a receiving function. Also, the display device 1 may not be a fixed type as shown in Figure 2, but may be a display device that can be moved by the user or is self-propelled.
[0041] (Example of display image) Figure 3 is a diagram showing an example of display image according to this embodiment. In the example shown in the upper part of Figure 3, the display image generated by the synthesis unit 50 and displayed by the display unit 60 includes a first area R1 for displaying image content and a second area R2 for displaying notification information. The second area R2 also includes a character CR included in the notification information, text data UC indicating the content of speech by the character CR, and text data UU indicating speech by the user (corresponding to "you" in Figure 3).
[0042] In the example shown in the lower part of Figure 3, the display image shown by the display unit 60 does not contain image content, but contains notification information. In the example shown in the lower part of Figure 3, the display image also includes a character CR included in the notification information, text data UC indicating the content of the speech made by the character CR, and text data UU indicating the speech made by the user.
[0043] <Information Processing Device 100> The information processing device 100 generates notification information based on the input information and content data. The information processing device 100 supplies the generated notification information to the synthesis unit 50.
[0044] Furthermore, the information processing apparatus 100 decodes content data. The information processing apparatus 100 supplies image content obtained by decoding the content data to the image quality control unit 30, and supplies audio content to the sound quality control unit 40.
[0045] Furthermore, the information processing apparatus 100 changes the output mode of the display apparatus 1. Therefore, the information processing apparatus 100 can also be referred to as an output mode changing apparatus. As an example, the output mode changed by the information processing apparatus 100 includes the image quality of an image output from the display unit 60 of the display apparatus 1. As another example, the output mode changed by the information processing apparatus 100 includes the sound quality of audio output from the speaker 70 of the display apparatus 1.
[0046] As an example, the information processing apparatus 100 changes the output mode of the display apparatus 1 by generating an image quality parameter and a sound quality parameter based on notification information and input information. The information processing apparatus 100 supplies the generated image quality parameter to the image quality control unit 30, and supplies the sound quality parameter to the sound quality control unit 40.
[0047] Returning to FIG. 1, the configuration of each unit of the information processing apparatus 100 included in the display apparatus 1 will be described. As shown in FIG. 1, the information processing apparatus 100 includes a first acquisition unit 11, a control unit 12, a second acquisition unit 21, and a content reproduction unit 22. Note that designations such as "first" and "second" do not limit the present embodiment. For example, either one or both of the "first acquisition unit 11" and the "second acquisition unit 21" may be simply expressed as an "acquisition unit".
[0048] (First Acquisition Unit 11) The first acquisition unit 11 acquires input information input to the display apparatus 1 from the input unit 10. As described above, examples of the input information include sensing data obtained by sensing a user who uses the display apparatus 1, sensing data obtained by sensing the periphery of the display apparatus 1 (for example, illuminance information indicating the brightness around the display apparatus 1), and an operation signal from a remote control device. As described above, an example of the sensing data obtained by sensing the user who uses the display apparatus 1 includes operation information indicating an operation by the user's voice. That is, the first acquisition unit 11 acquires operation information indicating an operation by the user's voice.
[0049] Further, among the input information acquired by the first acquisition unit 11, information indicating the user's reaction to notification information that notifies the user of a change in output mode from the display device 1 is also referred to as reaction information. That is, the first acquisition unit 11 acquires reaction information indicating the user's reaction to the notification information notified to the user from the display device 1 (output device).
[0050] As an example, the reaction information includes audio data including the user's utterance in response to the notification information. That is, the first acquisition unit 11 acquires reaction information including audio data including the user's utterance in response to the notification information.
[0051] As another example, the reaction information indicates an option selected by the user from among options related to the output mode notified to the user. That is, the first acquisition unit 11 acquires reaction information indicating the option selected by the user from among the options related to the output mode. Examples of the options related to the output mode will be described later.
[0052] (Control Unit 12) The control unit 12 executes analysis processing for analyzing at least any one of content data, image content, audio content, and input information, and performs various types of processing based on the analysis result.
[0053] As an example, the control unit 12 changes the output mode of the display device 1 based on the notification information and the reaction information. More specifically, the control unit 12 generates at least any one of an image quality parameter and a sound quality parameter based on the notification information and the reaction information. The control unit 12 supplies the generated image quality parameter to the image quality control unit 30. Further, the control unit 12 supplies the generated sound quality parameter to the sound quality control unit 40.
[0054] As another example, the control unit 12 generates notification information based on the input information. The control unit 12 supplies the generated notification information to the synthesis unit 50. Alternatively, as described above, the control unit 12 may be configured to supply the generated notification information to at least any one of the image quality control unit 30 or the sound quality control unit 40.
[0055] Specific examples of processing executed by the control unit 12 will be described later.
[0056] (Second acquisition unit 21) The second acquisition unit 21 acquires content data related to the content. For example, the second acquisition unit 21 acquires the content data received by the receiving unit 20 described above. The second acquisition unit 21 supplies the acquired content data to the content playback unit 22.
[0057] (Content playback unit 22) The content playback unit 22 controls the playback of the content indicated by the content data supplied from the second acquisition unit 21.
[0058] As an example, the content playback unit 22 decodes the encoded image contained in the content data and supplies the image content obtained by the decoding process to the image quality control unit 30. This image content constitutes at least a part of the display image that the display unit 60 displays.
[0059] Furthermore, the content playback unit 22 decodes the encoded audio data contained in the content data and supplies the audio content obtained through the decoding process to the sound quality control unit 40. This audio content constitutes at least a portion of the output audio output by the speaker 70.
[0060] Furthermore, the content playback unit 22 supplies content data and decoded content to the control unit 12.
[0061] (Specific Configuration of Information Processing Device 100) Next, with reference to Figure 4, a specific configuration example of the information processing device 100 according to this embodiment will be described. Figure 4 is a block diagram showing a specific configuration example of the information processing device 100 according to this embodiment. In the following description, redundant explanations of matters already described regarding the information processing device 100 may be omitted. Furthermore, specific examples of the processing performed by each part will be described later.
[0062] (Control Unit 12) As shown in Figure 4, the control unit 12 includes an analysis unit 13, an output mode control unit 14, a notification information generation unit 15, and an analysis unit 16.
[0063] (Analysis Unit 13) The analysis unit 13 analyzes the input information supplied from the first acquisition unit 11. The analysis unit 13 supplies the analysis results to at least one of the output mode control unit 14 and the notification information generation unit 15.
[0064] For example, if the input information is response information including voice data, the analysis unit 13 identifies the user's speech content by analyzing the voice data. If the input information is illuminance information indicating the brightness around the display device 1, the analysis unit 13 identifies the brightness around the display device 1 by analyzing the illuminance information. If the input information is an operation signal from a remote control device, the analysis unit 13 identifies the user's operation by analyzing the operation signal.
[0065] (Output Mode Control Unit 14) The output mode control unit 14 controls the output mode of the display device 1. More specifically, the output mode control unit 14 controls the output mode of the display device 1 by generating at least one of the image quality parameters of the display image displayed by the display device 1 and the sound quality parameters of the output sound output by the display device 1. The output mode control unit 14 supplies the generated image quality parameters to the image quality control unit 30. The output mode control unit 14 also supplies the generated sound quality parameters to the sound quality control unit 40.
[0066] Furthermore, as shown in Figure 4, the output mode control unit 14 includes a first control unit 141, a second control unit 142, and a fifth control unit 143.
[0067] (First control unit 141) The first control unit 141 changes the output mode of the display device 1 based on the notification information and the response information. For example, the first control unit 141 changes the output mode based on the response information indicating the option selected by the user from among the options for the output mode.
[0068] (Second control unit 142) The second control unit 142 changes the output mode of the display device 1 based on one or more setting pieces of information relating to a predetermined image included in the notification information. Setting information is information set by the user. For example, the second control unit 142 uses items worn by a character (clothes, accessories, shoes, etc.) included in the notification information as setting information and changes the output mode of the display device 1 based on the items worn by the character.
[0069] (Fifth control unit 143) The fifth control unit 143 causes the output mode of the display device 1 to differ depending on whether the display device 1 is outputting notification information or not. As an example, the fifth control unit 143 causes the content output mode to differ depending on whether the display device 1 is outputting notification information or not. As another example, the fifth control unit 143 causes the notification information output mode to differ depending on whether the display device 1 is outputting notification information or not.
[0070] (Notification Information Generation Unit 15) The notification information generation unit 15 generates notification information. More specifically, the notification information generation unit 15 generates notification images and notification sounds. The notification information generation unit 15 supplies the generated notification images and notification sounds to the synthesis unit 50.
[0071] (Prompt generation unit 151) The prompt generation unit 151 generates input data (prompts) to be input to the trained model LM based on the analysis results by the analysis unit 13. The prompt generation unit 151 also inputs the generated input data to the trained model LM. Here, the trained model LM may be configured as a server device 300 connected to the information processing device 100 via a network N, as shown in Figure 4, or it may be configured as the information processing device 100. The trained model LM may also be a large-scale language model, but is not limited to this. Any machine learning generative model can be used as the trained model LM.
[0072] The specific prompts generated by the prompt generation unit 151 are not limited to this embodiment, but as an example, the prompt generation unit 151 generates prompts that cause the trained model LM to generate further notification content based on the user's response information to the notification information. For example, the prompt generation unit 151 generates prompts that cause the trained model LM to generate further notification content based on the user's response to the notification information which includes character CR, and the content spoken by character CR. In other words, the prompt generation unit 151 generates prompts that cause the trained model LM to generate content that sounds like character CR is having a conversation with the user.
[0073] (Speech content acquisition unit 152) The speech content acquisition unit 152 acquires the speech content generated by the trained model LM based on the prompt generated by the prompt generation unit 151. The speech content acquisition unit 152 acquires the speech content in the form of text data, for example, but this is not limited to this embodiment.
[0074] (Third control unit 153) The third control unit 153 changes the frequency of outputting notification information according to the analysis results obtained by the analysis unit 16, which will be described later, by analyzing the content or content data. As described above, the notification information may be configured to be output at a predetermined timing, in which case the third control unit 153 increases or decreases the frequency of outputting the notification information. As an example, the third control unit 153 decreases the frequency of outputting notification information according to the analysis results.
[0075] (Fourth control unit 154) The fourth control unit 154 changes the output frequency of the notification information according to the brightness indicated by the illuminance information. Similar to the third control unit 153, the fourth control unit 154 also increases or decreases the frequency of output of the notification information when the notification information is output at a predetermined timing. For example, if the brightness indicated by the illuminance information is dimmer than a predetermined brightness, the fourth control unit 154 lowers the output frequency of the notification information.
[0076] (First generation unit 155) The first generation unit 155 generates notification information indicating options regarding the output mode. In other words, the first generation unit 155 generates notification information indicating options regarding image quality or sound quality.
[0077] As an example, the first generation unit 155 first selects an option from a predetermined set of options that corresponds to the user's utterance analyzed by the analysis unit 13. The first generation unit 155 supplies the selected option to the prompt generation unit 151, causing the prompt generation unit 151 to generate a prompt for generating notification information that allows the user to choose which option to select. The prompt generated by the prompt generation unit 151 is input to the trained model LM, and the utterance content acquisition unit 152 acquires the utterance content. The first generation unit 155 generates notification information that includes the utterance content acquired by the utterance content acquisition unit 152. The first generation unit 155 supplies the generated notification information to the voice generation unit 157 and the image generation unit 158.
[0078] Furthermore, if a predetermined image included in the notification information represents an object selected from a plurality of selectable objects, the first generation unit 155 will make the options indicated by the notification information different depending on the selected object. For example, if the plurality of objects are objects of different characters, the first generation unit 155 will make the representation of the options indicated by the notification information different depending on the selected object.
[0079] (Second generation unit 156) The second generation unit 156 generates a predetermined image based on one or more setting pieces of information relating to a predetermined image included in the notification information. For example, using multiple outfits of a character included in the notification information as setting pieces, the second generation unit 156 superimposes the images of the outfits selected by the user onto the image of the character generated by the image generation unit 158 (described later), thereby generating an image in which the character generated by the image generation unit 158 is wearing the outfit selected by the user. The second generation unit 156 supplies the generated image to the image generation unit 158.
[0080] Furthermore, the second generation unit 156 outputs multiple objects. For example, the second generation unit 156 generates an image that allows the user to select one of a plurality of objects that can be selected as a predetermined image, and the image includes the plurality of objects. The second generation unit 156 supplies the generated image to the synthesis unit 50.
[0081] Furthermore, the second generation unit 156 outputs one or more setting information. For example, the second generation unit 156 generates an image that allows the user to select at least one of one or more setting information relating to a predetermined image included in the notification information, and which includes one or more setting information. The second generation unit 156 supplies the generated image to the synthesis unit 50.
[0082] (Voice Generation Unit 157) The voice generation unit 157 generates voice data (notification voice) that indicates the speech content acquired by the speech content acquisition unit 152 or the speech content included in the notification information supplied from the first generation unit 155. Here, the voice generation unit 157 may determine the tone and accent of the speech in the voice data based on the analysis results of the analysis unit 13. Alternatively, the prompt generated by the prompt generation unit 151 may include instruction information to specify the tone, and the voice generation unit 157 may determine the tone and accent of the speech in the voice data based on the response of the trained model LM to the instruction. The voice generation unit 157 supplies the generated notification voice to the synthesis unit 50.
[0083] (Image generation unit 158) The image generation unit 158 generates a notification image. As an example, the image generation unit 158 generates a notification image including a character image based on the analysis results by the analysis unit 13. As another example, the image generation unit 158 generates a notification image including text data indicating the content of the speech contained in the notification information supplied from the first generation unit 155. As yet another example, the image generation unit 158 generates a notification image including an image generated by the second generation unit 156. The image generation unit 158 supplies the generated notification image to the synthesis unit 50.
[0084] Furthermore, as shown in Figure 3, the image generation unit 158 may be configured to generate not only the character CR but also a speech bubble image that displays the content of what the character CR says.
[0085] Furthermore, the image generation unit 158 may be configured to create images of at least a portion of the character using image generation AI. Alternatively, the image generation unit 158 may be configured to perform CG processing on images acquired from an external source by the input unit 10. Alternatively, the image generation unit 158 may generate images of the character using images created by image generation AI from text data related to speech.
[0086] (Analysis Unit 16) The analysis unit 16 analyzes the content or content data supplied from the content playback unit 22. "Analyzing content" refers to, but is not limited to, analyzing image content and audio content, and may also include analyzing related information. Related information may include various data specified by broadcast standards, data related to electronic program guides, and data obtained from the web through searches etc. based on data obtained from broadcasts. An example of the analysis result is the content category, such as movies and dramas.
[0087] (Processing flow by the information processing device 100 1) An example of the processing flow by the information processing device 100 will be explained with reference to Figure 5. Figure 5 is a flowchart showing an example of the processing flow by the information processing device 100 according to this embodiment.
[0088] (Step S11) In step S11, the notification information generation unit 15 supplies notification information (at least one of a notification image and a notification sound) to the synthesis unit 50. The synthesis unit 50 generates at least one of a display image and an output sound, and the notification information is output from the display device 1 when the display image is output from the display unit 60 or the output sound is output from the speaker 70.
[0089] (Step S12) In step S12, the first acquisition unit 11 acquires reaction information indicating the user's response to the notification information that was notified to the user from the display device 1 in step S11. The first acquisition unit 11 supplies the acquired reaction information to the control unit 12.
[0090] (Step S13) In step S13, the analysis unit 13 of the control unit 12 analyzes the reaction information supplied from the first acquisition unit 11 in step S12. The analysis unit 13 supplies the analysis results to the output mode control unit 14.
[0091] (Step S14) In step S14, the first control unit 141 of the output mode control unit 14 changes the output mode of the display device 1 based on the notification information in step S11 and the analysis results supplied from the analysis unit 13 in step S13.
[0092] (Specific Processing Example 1) A specific processing example of the process shown in Figure 5 will be explained with reference to Figure 6. Figure 6 is a flowchart showing the specific processing flow by the information processing device 100 according to this embodiment.
[0093] (Step S21) In step S21, the notification information generation unit 15 supplies notification information (at least one of a notification image and a notification sound) to the synthesis unit 50. The synthesis unit 50 generates at least one of a display image and an output sound, and the notification information is output from the display device 1 when the display image is output from the display unit 60 or the output sound is output from the speaker 70.
[0094] As described above, the notification information may be configured to be output in response to user operations, or it may be configured to be output at a predetermined timing. In other words, step S21 may be executed in response to user operations, or it may be executed at a predetermined timing.
[0095] (Step S22) In step S22, the first acquisition unit 11 acquires input information indicating that the user wants to change the output mode. The first acquisition unit 11 supplies the acquired input information to the control unit 12. As an example, as shown in the lower part of Figure 3, the first acquisition unit 11 acquires input information including voice data indicating that the user has spoken "I want to change the image quality".
[0096] (Step S23) In step S23, the analysis unit 13 of the control unit 12 analyzes the input information supplied from the first acquisition unit 11 in step S22. The analysis unit 13 supplies the analysis results to the notification information generation unit 15. For example, if the input information includes audio data indicating "I want to change the image quality", the analysis unit 13 analyzes the audio data to determine that the user has made an utterance indicating that they want to change the image quality.
[0097] (Step S24) In step S24, the first generation unit 155 of the notification information generation unit 15 generates notification information indicating options regarding the output mode based on the analysis results in step S23. More specifically, in step S24, the following steps S241 to S244 are executed.
[0098] (Step S241) In step S241, the first generation unit 155 of the notification information generation unit 15 selects an option based on the analysis result in step S23. For example, if the analysis result is that the user has made an utterance indicating that they want to change the image quality, the first generation unit 155 selects "bright colors" and "subdued colors" as options. The first generation unit 155 supplies the selected options to the prompt generation unit 151.
[0099] (Step S242) In step S242, the prompt generation unit 151 generates a prompt to generate notification information that allows the user to select one of the options supplied by the first generation unit 155 in step S241. The prompt generation unit 151 inputs the generated prompt to the trained model LM.
[0100] (Step S243) In step S243, the speech content acquisition unit 152 acquires the speech content output from the trained model LM when the prompt generation unit 151 inputs a prompt to the trained model LM in step S242. The speech content acquisition unit 152 supplies the acquired speech content to the first generation unit 155.
[0101] (Step S244) In step S244, the first generation unit 155 generates notification information including the speech content acquired by the speech content acquisition unit 152 in step S243. The first generation unit 155 supplies the generated notification information to the voice generation unit 157 and the image generation unit 158.
[0102] (Step S25) In step S25, the voice generation unit 157 generates voice data (notification voice) that indicates the content of the speech included in the notification information supplied from the first generation unit 155 in step S244. The voice generation unit 157 supplies the generated notification voice to the synthesis unit 50.
[0103] Furthermore, in step S25, the image generation unit 158 generates a notification image that includes text data indicating the content of the speech contained in the notification information supplied from the first generation unit 155 in step S244. The image generation unit 158 supplies the generated notification image to the synthesis unit 50.
[0104] An example of an image displayed on the display unit 60 in step S25 will be explained with reference to Figure 7. Figure 7 is a diagram showing an example of an image displayed on the display unit 60 according to this embodiment. As an example, in step S25, as shown in the upper part of Figure 7, an image is displayed on the display unit 60 in which character CR speaks text data UC "Do you like bright colors or subdued colors?" which indicates the content of the speech that prompts the user to choose between "bright colors" and "subdued colors".
[0105] (Step S26) In step S26, the first acquisition unit 11 acquires response information indicating the option selected by the user in response to the notification information output in step S25. The first acquisition unit 11 supplies the acquired response information to the control unit 12.
[0106] As an example, in step S26, the first acquisition unit 11 acquires reaction information indicating that the user has selected "bold colors," as shown in the upper part of Figure 7.
[0107] (Step S27) In step S27, the analysis unit 13 of the control unit 12 analyzes the reaction information supplied from the first acquisition unit 11 in step S26. The analysis unit 13 supplies the analysis results to the output mode control unit 14.
[0108] For example, if the reaction information supplied from the first acquisition unit 11 in step S26 is reaction information indicating that the user selected "bold colors," the analysis unit 13 will identify that the user selected "bold colors."
[0109] (Step S28) In step S28, the first control unit 141 of the output mode control unit 14 changes the output mode based on the analysis results in step S27. That is, the first control unit 141 generates at least one of the image quality parameters for the display image to be displayed by the display device 1, and the sound quality parameters for the output sound to be output by the display device 1, based on the selection made by the user. The first control unit 141 changes the output mode of the display image by supplying the generated image quality parameters to the image quality control unit 30. The first control unit 141 also changes the output mode of the output sound by supplying the generated sound quality parameters to the sound quality control unit 40.
[0110] For example, if the analysis results show a tendency toward the output configuration desired by the user, the first control unit 141 changes the output configuration to match the tendency toward the output configuration desired by the user.
[0111] As a more specific example, if the analysis results indicate that the user has selected "vivid colors," the first control unit 141 generates image quality parameters that make the colors of the display image more vibrant. For example, the first control unit 141 generates image quality parameters that are equal to the value of the color intensity parameter plus 3.
[0112] As another example, if the analysis results indicate that the user has selected "calm color scheme," the first control unit 141 generates image quality parameters that result in calmer colors for the displayed image. For example, the first control unit 141 generates image quality parameters that are three times the value of the color intensity parameter.
[0113] As yet another example, the notification information output in step S25 includes at least one of a predetermined image and audio data including speech mediated by the predetermined image, and in step S26, the first acquisition unit 11 acquires response information including audio data including speech by the user in response to the notification information. In this case as well, the first control unit 141 changes the output mode based on the notification information and the audio data including speech by the user in response to the notification information. With this configuration, the user can change the output mode by conversing with a character included in the predetermined image, so the information processing device 100 can be easily adjusted to the output mode desired by the user without requiring the user to perform complicated operations.
[0114] As another example, the first control unit 141 changes the output mode of the notification information, along with the output mode of the content. For example, as described above, when the information processing device 100 supplies notification information to the image quality control unit 30 and the sound quality control unit 40, the first control unit 141 instructs the image quality control unit 30 and the sound quality control unit 40 to change the output mode of the notification information. More specifically, the first control unit 141 instructs the image quality control unit 30 to apply the generated image quality parameters to the notification image, and instructs the sound quality control unit 40 to apply the generated sound quality parameters to the notification sound. With this configuration, the information processing device 100 changes, for example, the image quality of the character CR, the background image of the character CR, and the sound quality shown in the lower part of Figure 7, so that the user can easily understand how the image quality and sound quality have been changed.
[0115] (Examples of output mode options) Examples of output mode options for display images include, in addition to the above-mentioned "bright colors" and "subdued colors," "images that emphasize contrast" and "images that emphasize gradation expression," "clear and sharp images" and "subdued images with reduced noise," as well as "warm-toned images" and "cool-toned images."
[0116] For example, if the user selects "image with emphasized contrast," the first control unit 141 generates an image quality parameter with a value equal to +3 of the contrast parameter value. On the other hand, if the user selects "image with emphasis on gradation expression," the first control unit 141 generates an image quality parameter with a value equal to -3 of the contrast parameter value.
[0117] Furthermore, if the user selects "clear and sharp image," the first control unit 141 generates an image quality parameter with a value equal to +3 of the sharpness parameter value. On the other hand, if the user selects "softer image with reduced noise," the first control unit 141 generates an image quality parameter with a value equal to -3 of the sharpness parameter value.
[0118] Furthermore, if "warm-toned video" is selected by the user, the first control unit 141 generates an image quality parameter with a value obtained by adding 3 to the color temperature parameter. On the other hand, if "cool-toned video" is selected by the user, the first control unit 141 generates an image quality parameter with a value obtained by subtracting 3 from the color temperature parameter.
[0119] Furthermore, examples of options regarding the output method of the output audio include "standard audio with consistent volume levels" and "clear audio with easily audible dialogue."
[0120] If the user selects "standard voice with uniform volume," the first control unit 141 generates sound quality parameters with the same output value in each frequency band. On the other hand, if the user selects "clear voice with easy-to-understand dialogue," the first control unit 141 generates sound quality parameters in which the output in the high and low frequency bands is greater than in the other frequency bands.
[0121] (Specific Processing Example 2) As another example of specific processing by the information processing device 100, we will describe a process in which step S24 is executed again after step S27 described above.
[0122] (Steps S21 to S26) Steps S21 to S26 are performed in the same manner as in the example described above.
[0123] (Step S27) In step S27, the analysis unit 13 of the control unit 12 analyzes the reaction information supplied from the first acquisition unit 11 in step S26. The analysis unit 13 supplies the analysis results to the output mode control unit 14 or the control unit 12 according to the analysis results.
[0124] For example, if the analysis unit 13 determines in step S27 that the user has selected "bold colors," the analysis unit 13 supplies the analysis results to the control unit 12. On the other hand, if the analysis unit 13 determines in step S27 that the user has selected "subdued colors," the analysis unit 13 supplies the analysis results to the output mode control unit 14, and step S28 described above is executed.
[0125] (Step S24) If the analysis results are supplied to the control unit 12 in step S27, step S24 is executed again. In step S24, which is executed again, the first generation unit 155 of the notification information generation unit 15 generates notification information indicating options regarding the output mode based on the analysis results in step S27. Specifically, the following step S241 is executed.
[0126] (Step S241) In step S241, the first generation unit 155 selects an option based on the analysis result in step S27. For example, if the analysis result in step S27 indicates that the user has selected "vivid colors", the first generation unit 155 selects "clear and sharp image" and "subdued image with reduced noise" as options. The first generation unit 155 supplies the selected options to the prompt generation unit 151. The processing from step S242 onward is the same as the processing described above.
[0127] In other words, if a user selects "bold colors," it is likely that the user prefers images with emphasized outlines and borders, resulting in a sharper image. Thus, the information processing device 100 can easily adjust to the user's desired output by presenting further options based on the user's selection.
[0128] (Effects of the display device 1) As described above, the display device 1 according to this embodiment includes a first acquisition unit 11 that acquires response information indicating the user's reaction to the notification information notified to the user from the display device 1, and a first control unit 141 that changes the output mode of the display device 1 based on the notification information and the response information. With this configuration, the display device 1 can be easily adjusted to the output mode desired by the user.
[0129] Furthermore, the display device 1 generates notification information indicating options for output modes, and changes the output mode based on the option selected by the user. With this configuration, the display device 1 can be easily adjusted to the output mode desired by the user without requiring the user to perform complicated operations.
[0130] (Processing flow by the information processing device 100 2) Another example of the processing flow by the information processing device 100 will be described with reference to Figure 8. Figure 8 is a flowchart showing another example of the processing flow by the information processing device 100 according to this embodiment. In this example, an example of the process by which the information processing device 100 changes the output frequency of notification information will be described.
[0131] (Step S31) In step S31, the second acquisition unit 21 acquires content data. The second acquisition unit 21 supplies the acquired content data to the content playback unit 22. The content playback unit 22 decodes the content data and supplies the content data or the decoded content to the control unit 12.
[0132] (Step S32) In step S32, the analysis unit 16 of the control unit 12 analyzes the content or content data supplied from the content playback unit 22 in step S31. For example, the analysis unit 16 identifies the category of the content. The analysis unit 16 supplies the analysis results to the notification information generation unit 15.
[0133] (Step S33) In step S33, the third control unit 153 of the notification information generation unit 15 changes the output frequency of the notification information according to the analysis results in step S32. As an example, the third control unit 153 changes the output frequency of the notification information according to the category of the content. For example, if the analysis results in step S32 indicate that the content is a movie or a drama, the third control unit 153 lowers the output frequency of the notification information.
[0134] Thus, for example, if the display device 1 is outputting content that users tend to watch intently, the display device 1 will reduce the frequency of outputting notification information. Therefore, the display device 1 can output notification information in a way that does not interfere with the user's viewing of the content.
[0135] (Processing flow by the information processing device 100 3) Another example of the processing flow by the information processing device 100 will be described with reference to Figure 9. Figure 9 is a flowchart showing yet another example of the processing flow by the information processing device 100 according to this embodiment. In this example, another example of the process by which the information processing device 100 changes the output frequency of the notification information will be described.
[0136] (Step S31A) In step S31A, the first acquisition unit 11 acquires sensing data (illuminance information) that senses the illuminance around the display device 1. The first acquisition unit 11 supplies the acquired sensing data to the control unit 12.
[0137] (Step S32A) In step S32A, the analysis unit 13 of the control unit 12 determines the brightness around the display device 1 by analyzing the sensing data. The analysis unit 13 supplies information indicating the determined brightness to the notification information generation unit 15.
[0138] (Step S33A) In step S33A, the fourth control unit 154 of the notification information generation unit 15 changes the output frequency of the notification information according to the analysis result in step S32A. For example, if the analysis result indicates that the brightness around the display device 1 is darker than a predetermined brightness, the fourth control unit 154 lowers the output frequency of the notification information.
[0139] Thus, for example, in an environment where users tend to concentrate on viewing content, the display device 1 reduces the frequency of outputting notification information. Therefore, even in this configuration, the display device 1 can output notification information in a way that does not interfere with the user's viewing of content.
[0140] (Processing flow by the information processing device 100 4) Another example of the processing flow by the information processing device 100 will be explained with reference to Figure 10. Figure 10 is a flowchart showing yet another example of the processing flow by the information processing device 100 according to this embodiment. In this example, an example of processing that makes the output pattern different depending on whether the display device 1 is outputting notification information or not will be explained.
[0141] (Step S41) In step S41, the notification information generation unit 15 supplies notification information (at least one of a notification image and a notification sound) to the synthesis unit 50. The synthesis unit 50 generates at least one of a display image and an output sound, which are output from the display unit 60 or the speaker 70.
[0142] As described above, the notification information may be configured to be output in response to user operations, or it may be configured to be output at any frequency regardless of user operations. In other words, step S41 may be executed in response to user operations, or it may be executed at a predetermined timing.
[0143] (Step S42) In step S42, the fifth control unit 143 changes the output mode of the display device 1 because notification information was output in step S41.
[0144] As an example, as shown in the upper part of Figure 3, if the display image includes a first area R1 for displaying image content and a second area R2 for displaying notification information, the fifth control unit 143 darkens the first area R1. Furthermore, the fifth control unit 143 may brighten the second area R2.
[0145] As another example, if an audio output is output as notification information in step S41, the fifth control unit 143 reduces the volume of the audio content. Furthermore, the fifth control unit 143 may increase the volume of the audio output for the notification information.
[0146] (Step S43) In step S43, the notification information generation unit 15 terminates the output of notification information from the display unit 60 or speaker 70 by ending the supply of notification information (at least one of notification image and notification sound) to the synthesis unit 50.
[0147] For example, if the first acquisition unit 11 does not acquire user input information (response information) for the notification information for a predetermined period of time, or if it acquires an operation signal from the remote control device indicating an operation to terminate the output of the notification information, the notification information generation unit 15 will terminate supplying the notification information to the synthesis unit 50.
[0148] (Step S44) In step S44, the fifth control unit 143 changes the output mode of the display device 1 because the output of the notification information has been completed in step S43.
[0149] As an example, the fifth control unit 143 restores the output mode that was changed in step S42. For example, if the first region R1 shown in the upper part of Figure 3 is darkened in step S42, the fifth control unit 143 restores the brightness of the first region R1 to the brightness it was at before step S42 was executed. Also, if the volume of the audio content is reduced in step S42, the fifth control unit 143 restores the volume of the audio content to the brightness it was at before step S42 was executed.
[0150] Thus, the information processing device 100 outputs different information depending on whether the display device 1 is outputting notification information or not. Therefore, when the information processing device 100 outputs notification information, it can output the notification information in a way that is easy for the user to see or hear.
[0151] Furthermore, the information processing device 100 may perform other processing in place of, or in addition to, the processing that differentiates the output mode depending on whether or not notification information is being output. For example, the first acquisition unit 11 may be configured to acquire operation information related to the output mode, which indicates the user's voice operation, when notification information is being output, and not acquire operation information related to the output mode when notification information is not being output. With this configuration, the information processing device 100 does not process any information from the user other than the user's response to the notification information, thus reducing the processing load.
[0152] (Processing flow by the information processing device 100 5) Another example of the processing flow by the information processing device 100 will be described with reference to Figure 11. Figure 11 is a flowchart showing yet another example of the processing flow by the information processing device 100 according to this embodiment. In this example, an example of processing that changes the output mode based on setting information related to a predetermined image will be described.
[0153] (Step S51) In step S51, the second generation unit 156 outputs one or more setting information. As an example, the second generation unit 156 generates an image that allows the user to select at least one of one or more setting information relating to a predetermined image included in the notification information. The second generation unit 156 outputs one or more setting information by supplying the generated image to the synthesis unit 50.
[0154] An example of an output image produced in step S51 will be explained with reference to Figure 12. Figure 12 is a diagram showing another example of an image displayed in the display unit 60 according to this embodiment. As shown in Figure 12, the output image produced in step S51 includes setting information, which is multiple outfits for the character. In the output image shown in Figure 12, any one of the multiple outfits for the character can be selected.
[0155] (Step S52) In step S52, the first acquisition unit 11 acquires input information indicating the selected setting information from among the one or more setting information outputs in step S51. The first acquisition unit 11 supplies the acquired input information to the control unit 12.
[0156] (Step S53) In step S53, the analysis unit 13 of the control unit 12 analyzes the input information supplied from the first acquisition unit 11 in step S52. The analysis unit 13 supplies the analysis results to the output mode control unit 14.
[0157] (Step S54) In step S54, the second control unit 142 of the output mode control unit 14 changes the output mode of the display device 1 based on the analysis results in step S53.
[0158] For example, if the analysis result in step S53 indicates that the setting information included in the selection group CG1 among the setting information included in the output image shown in Figure 12 has been selected, the second control unit 142 generates image quality parameters that make the colors of the display image more vibrant. In other words, if clothing with bright colors is selected as the character's clothing, the second control unit 142 generates image quality parameters that make the colors of the display image more vibrant.
[0159] On the other hand, if the analysis result in step S53 indicates that the setting information included in the selection group CG2 among the setting information included in the output image shown in Figure 12 has been selected, the second control unit 142 generates image quality parameters that result in a subdued color for the display image. In other words, if clothing with subdued colors is selected as the character's clothing, the second control unit 142 generates image quality parameters that result in a subdued color for the display image.
[0160] As another example, if the analysis result in step S53 indicates that a garment with a color close to a primary color has been selected from among the character's multiple garments, the second control unit 142 generates image quality parameters that make the display image darker. On the other hand, if the analysis result in step S53 indicates that a garment with a light or muted color has been selected from among the character's multiple garments, the second control unit 142 generates image quality parameters that make the display image lighter.
[0161] In addition, the character's background color may be selected instead of the character's clothing, or in addition to the character's clothing. In this case, the second control unit 142 generates image quality parameters that make the color of the display image suitable for the character's background color. For example, if the lighting in the room where the television is viewed is warm-toned, and the user selects a warm-toned color as the character's background color, the image quality parameters will be set to values suitable for warm-toned lighting. Note that the selection may not be the color itself, such as orange, but rather the color temperature. In this way, even if the information processing device 100 does not acquire illuminance information, it can output a display image suitable for the color temperature of the lighting based on the character's background color. In other words, even if the information processing device 100 does not have a sensor for acquiring illuminance information, it can generate a display image that takes into account the color temperature of the room's lighting.
[0162] In this manner, the information processing device 100 changes the output mode of the display device 1 based on one or more setting pieces of information relating to a predetermined image included in the notification information. With this configuration, the information processing device 100 changes to the user's preferred output mode estimated from the image set by the user, so the user can easily adjust to their desired output mode.
[0163] (Processing flow by the information processing device 100 6) Further examples of the processing flow by the information processing device 100 will be described with reference to Figure 13 and Figure 6 described above. Figure 13 is a flowchart showing yet another example of the processing flow by the information processing device 100 according to this embodiment. In this example, an example of processing that makes the options indicated by the notification information different depending on the object selected as a predetermined image will be described.
[0164] (Step S61) In step S61, the second generation unit 156 outputs multiple objects. As an example, the second generation unit 156 generates an image that allows the user to select one of multiple objects that can be selected as a predetermined image. The second generation unit 156 outputs the generated image from the display unit 60 by supplying the generated image to the synthesis unit 50.
[0165] (Step S62) In step S62, the first acquisition unit 11 acquires input information indicating the selected object from among the multiple objects output in step S61. The first acquisition unit 11 supplies the acquired input information to the control unit 12.
[0166] (Step S63) In step S63, the analysis unit 13 of the control unit 12 analyzes the input information supplied from the first acquisition unit 11 in step S62. The analysis unit 13 supplies the analysis results to the notification information generation unit 15.
[0167] (Step S64) In step S64, the image generation unit 158 of the notification information generation unit 15 modifies the object based on the analysis results.
[0168] For example, if the analysis results indicate that the character CR shown in Figure 3 has been selected as the predetermined image, the image generation unit 158 generates a notification image that includes the character CR.
[0169] Further examples will be explained with reference to Figure 14. Figure 14 shows yet another example of an image displayed in the display unit 60 according to this embodiment.
[0170] As another example, if the analysis results indicate that a character CR that appears to have more detailed image quality than the character CR shown in Figure 3 has been selected as the predetermined image, as shown in Figure 14, the image generation unit 158 generates a notification image that includes the character CR.
[0171] Next, we will refer to Figure 6 again for further explanation.
[0172] (Step S21) In step S21, the notification information generation unit 15 supplies notification information (at least one of a notification image and a notification sound) to the synthesis unit 50. Here, the notification information generation unit 15 supplies the notification image generated in step S64 described above to the synthesis unit 50. The synthesis unit 50 generates at least one of a display image and an output sound, and the display image is output from the display unit 60, or the output sound is output from the speaker 70, thereby outputting the notification information from the display device 1.
[0173] (Step S22) In step S22, similar to the process described above, the first acquisition unit 11 acquires input information indicating that the output mode should be changed. The first acquisition unit 11 supplies the acquired input information to the control unit 12.
[0174] (Step S23) In step S23, similar to the process described above, the analysis unit 13 of the control unit 12 analyzes the input information supplied from the first acquisition unit 11 in step S22. The analysis unit 13 supplies the analysis results to the notification information generation unit 15.
[0175] (Step S24) In step S24, the first generation unit 155 of the notification information generation unit 15 generates notification information indicating options regarding the output mode based on the analysis results in step S23. More specifically, steps S241 to S244 below are executed.
[0176] (Step S241) In step S241, the first generation unit 155 of the notification information generation unit 15 selects an option based on the analysis result in step S63 and the analysis result in step S23. For example, if the analysis result in step S23 indicates that the user has made a statement indicating that they wish to change the image quality, and the analysis result in step S63 indicates that the character CR shown in Figure 14 has been selected as the predetermined image, the first generation unit 155 selects "Recommend increasing the color intensity from the current setting" and "Do not change the setting" as options. The first generation unit 155 supplies the selected options to the prompt generation unit 151.
[0177] (Step S242) In step S242, similar to the process described above, the prompt generation unit 151 generates a prompt to generate notification information that allows the user to select one of the options supplied by the first generation unit 155 in step S241. The prompt generation unit 151 inputs the generated prompt to the trained model LM.
[0178] (Step S243) In step S243, similar to the process described above, the speech content acquisition unit 152 acquires the speech content output from the trained model LM when the prompt generation unit 151 inputs a prompt to the trained model LM in step S242. The speech content acquisition unit 152 supplies the acquired speech content to the first generation unit 155.
[0179] (Step S244) In step S244, the first generation unit 155 generates notification information including the speech content acquired by the speech content acquisition unit 152 in step S243. The first generation unit 155 supplies the generated notification information to the voice generation unit 157 and the image generation unit 158.
[0180] (Processing from step S25 onwards) The processing from step S25 onwards is the same as the processing described above, so the explanation is omitted.
[0181] In this configuration, the image displayed on the display unit 60 in step S25 will be explained again with reference to Figure 14. As an example, in step S25, as shown in the upper part of Figure 14, an image of character CR speaking the following text data UC, which indicates the content of the speech that prompts the user to choose between the options "I recommend increasing the color intensity from the current setting" and "Do not change the setting," is displayed on the display unit 60.
[0182] In this way, the information processing device 100 generates notification information indicating that a predetermined image is the selected object, and the options indicated by the notification information differ depending on the selected object. For example, if an object is selected that the user does not seem to know much about image quality, the information processing device 100 selects options related to general settings, and if an object is selected that the user seems to know a lot about image quality, it selects options related to detailed settings. With this configuration, the information processing device 100 can estimate how knowledgeable the user is about image quality from the selected object and notify the user of options tailored to their needs.
[0183] [Example of implementation by software] The functions of the information processing device 100 (hereinafter referred to as "device") can be realized by a program that causes the device to function as a computer, and by a program that causes each control block of the device (particularly each part included in the control unit 12) to function as a computer.
[0184] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, the functions described in each of the embodiments are realized.
[0185] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.
[0186] Furthermore, some or all of the functions of each of the above control blocks can also be realized by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of the present invention. In addition, it is also possible to realize the functions of each of the above control blocks by, for example, a quantum computer.
[0187] Furthermore, each process described in the above embodiments may be performed by AI (Artificial Intelligence). In this case, the AI may operate on the control device described above, or it may operate on another device (for example, an edge computer or a cloud server).
[0188] Furthermore, the display device 1 may be implemented as a terminal device. The terminal device may be, for example, a smartphone, smartwatch, personal computer, or tablet owned by the user. The terminal device may have a display unit (the display unit 60 described above) that displays the dialogue between the user and the interactive agent. The information processing device 100 may, for example, display the dialogue between the user and the interactive agent on the display unit of the terminal device in the same display format as the display unit 60 of the display device 1.
[0189] [Summary] An information processing device (100) according to aspect 1 of the present invention includes a first acquisition unit (11) that acquires reaction information indicating the user's response to notification information notified to the user from an output device (display device 1, display unit 60, speaker 70), and a first control unit (141) that changes the output mode of the output device based on the notification information and the reaction information.
[0190] According to the above configuration, the information processing device can easily adjust the output mode of the output device to the output mode desired by the user.
[0191] In the information processing apparatus according to aspect 2 of the present invention, the notification information in aspect 1 includes at least one of a predetermined image and audio data including speech mediated by the predetermined image, the first acquisition unit acquires the response information including audio data including speech by the user in response to the notification information, and the first control unit may change the output mode based on the audio data included in the notification information and the response information.
[0192] According to the above configuration, the information processing device allows the user to change the output mode by interacting with a predetermined image, thus easily adjusting the output mode of the output device to the user's desired output mode without requiring the user to perform complicated operations.
[0193] An information processing device according to aspect 3 of the present invention further comprises a first generation unit (155) that generates notification information indicating the options for the output mode, the first acquisition unit acquires the response information indicating the options selected by the user, and the first control unit may change the output mode based on the options selected by the user.
[0194] With the above configuration, the information processing device can be easily adjusted to the output format desired by the user without requiring the user to perform complicated operations.
[0195] In the information processing device according to aspect 4 of the present invention, the predetermined image in aspect 3 represents an object selected from a plurality of selectable objects, and the first generation unit may make the selections indicated by the notification information differ depending on the selected object.
[0196] According to the above configuration, the information processing device can estimate how familiar the user is with the output format from the selected object and inform the user of options tailored to their needs.
[0197] In the information processing apparatus according to aspect 5 of the present invention, in any of aspects 2 to 4 above, a second generation unit (156) that generates the predetermined image based on one or more setting pieces of information relating to the predetermined image, and a second control unit (142) that changes the output mode based on the one or more setting pieces of information, may further be included.
[0198] According to the above configuration, the information processing device changes the output mode to the user's preferred mode, which is estimated from the image set by the user, so it can be easily adjusted to the output mode desired by the user.
[0199] The information processing device according to embodiment 6 of the present invention may further include a second acquisition unit (21) for acquiring content data relating to content, in any of embodiments 1 to 5 above.
[0200] According to the above configuration, the information processing device can change the output mode of the content data.
[0201] In the information processing device according to embodiment 7 of the present invention, the first control unit in embodiment 6 may change the output mode of the notification information, along with the output mode of the content, as the output mode.
[0202] According to the above configuration, the information processing device can make it easy for the user to understand how the output format of the notification information has been changed by changing the output format of the notification information.
[0203] An information processing device according to embodiment 8 of the present invention may further include a third control unit (153) that changes the output frequency of the notification information according to the analysis results of the content obtained by analyzing the content or the content data, as described in embodiment 6 or 7 above.
[0204] According to the above configuration, the information processing device can output notification information in a way that does not interfere with the user's viewing of content when the content is one that the user is intently watching.
[0205] In the information processing apparatus according to embodiment 9 of the present invention, in any of embodiments 1 to 8, the first acquisition unit may further include a fourth control unit (154) that acquires illuminance information indicating the brightness around the output device and changes the output frequency of the notification information according to the brightness indicated by the illuminance information.
[0206] With the above configuration, in environments where users are concentrating on viewing content, notification information can be output in a way that does not interfere with the user's viewing of the content.
[0207] An information processing device according to embodiment 10 of the present invention may further include a fifth control unit (143) that causes the output device to output different information depending on whether it is outputting the notification information or not, in any of embodiments 1 to 9.
[0208] According to the above configuration, when the information processing device outputs notification information, it can output the notification information in a way that is easy for the user to see or hear.
[0209] In the information processing apparatus according to embodiment 11 of the present invention, the first acquisition unit in any of embodiments 1 to 10 acquires operation information indicating user voice operation, and if the notification information is output, the first acquisition unit acquires the operation information relating to the output mode, and if the notification information is not output, it is not necessary to acquire the operation information relating to the output mode.
[0210] According to the above configuration, the information processing device can reduce its processing load.
[0211] In the information processing apparatus according to embodiment 12 of the present invention, the output mode in any of embodiments 1 to 11 may include the image quality of the image output from the output device.
[0212] According to the above configuration, the information processing device can be easily adjusted to the image quality desired by the user.
[0213] The information processing apparatus according to embodiment 13 of the present invention is equipped with the output device described in any of embodiments 1 to 12 above.
[0214] According to the above configuration, an information processing device can be realized that possesses the effects of the information processing device described above and is equipped with an output device.
[0215] A television receiver according to embodiment 14 of the present invention is equipped with the information processing device described in embodiment 13.
[0216] According to the above configuration, a television receiver that exhibits the effects of the information processing device described above can be realized.
[0217] A display control system (1) according to aspect 15 of the present invention includes a receiving device (receiving unit 20) that receives content data relating to content, an output device (display unit 60, speaker 70) that outputs the content, an acquisition device (input unit 10) that acquires response information indicating the user's response to notification information notified to the user by the output device, and an output mode changing device (information processing device 100) that changes the output mode of the content by the output device based on the notification information and the response information.
[0218] With the above configuration, the display control system can easily adjust the output mode of the output device to the output mode desired by the user.
[0219] An information processing method according to aspect 16 of the present invention includes a first acquisition step of acquiring reaction information indicating the user's response to notification information notified to the user from an output device, and a control step of changing the output mode of the output device based on the notification information and the reaction information.
[0220] According to the above configuration, the information processing method allows the output mode of the output device to be easily adjusted to the output mode desired by the user.
[0221] Each aspect of the present invention may be implemented by a computer, in which case the information processing program for the information processing device that enables the computer to implement the information processing device by operating the computer as each part (software element) of the information processing device, and a computer-readable recording medium on which the program is recorded, also fall within the scope of the present invention.
[0222] With the above configuration, the information processing program can easily adjust the output mode of the output device to the output mode desired by the user.
[0223] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.
[0224] 1 Display device (output device) 10 Input unit (acquisition device) 11 First acquisition unit 12 Control unit 13 Analysis unit 14 Output mode control unit 15 Notification information generation unit 20 Receiving unit (receiving device) 30 Image quality control unit 40 Sound quality control unit 50 Synthesis unit 60 Display unit (output device) 70 Speaker (output device) 100 Information processing device (output mode changing device) 141 First control unit 142 Second control unit 143 Fifth control unit 153 Third control unit 154 Fourth control unit 155 First generation unit 156 Second generation unit 300 Server device
Claims
1. An information processing device comprising: a first acquisition unit that acquires response information indicating the user's reaction to notification information notified to the user from an output device; and a first control unit that changes the output mode of the output device based on the notification information and the response information.
2. The information processing apparatus according to claim 1, wherein the notification information includes at least one of a predetermined image and audio data including speech mediated by the predetermined image, the first acquisition unit acquires the response information including audio data including speech by the user in response to the notification information, and the first control unit changes the output mode based on the audio data included in the notification information and the response information.
3. The information processing apparatus according to claim 2, further comprising a first generation unit that generates notification information indicating the options for the output mode, the first acquisition unit acquires the response information indicating the options selected by the user, and the first control unit changes the output mode based on the options selected by the user.
4. The information processing apparatus according to claim 3, wherein the predetermined image represents an object selected from a plurality of selectable objects, and the first generation unit causes the selection indicated by the notification information to differ depending on the selected object.
5. The information processing apparatus according to claim 2, further comprising: a second generation unit that generates the predetermined image based on one or more setting pieces of information relating to the predetermined image; and a second control unit that changes the output mode based on the one or more setting pieces of information.
6. The information processing apparatus according to claim 1, further comprising a second acquisition unit for acquiring content data relating to content.
7. The information processing apparatus according to claim 6, wherein the first control unit changes the output mode, along with the output mode of the content, the output mode of the notification information.
8. The information processing apparatus according to claim 6, further comprising a third control unit that changes the output frequency of the notification information according to the analysis results relating to the content obtained by analyzing the content or the content data.
9. The information processing apparatus according to claim 1, wherein the first acquisition unit further acquires illuminance information indicating the brightness around the output device, and further comprises a fourth control unit that changes the output frequency of the notification information according to the brightness indicated by the illuminance information.
10. The information processing apparatus according to claim 1, further comprising a fifth control unit that causes the output mode to differ depending on whether the output device is outputting the notification information or not.
11. The information processing apparatus according to claim 1, wherein the first acquisition unit acquires operation information indicating user voice operation, the first acquisition unit acquires the operation information relating to the output mode when outputting the notification information, and does not acquire the operation information relating to the output mode when not outputting the notification information.
12. The information processing apparatus according to claim 1, wherein the output mode includes the image quality of the image output from the output device.
13. An information processing device according to any one of claims 1 to 12, comprising the output device.
14. A television receiver equipped with the information processing device described in claim 13.
15. A display control system comprising: a receiving device for receiving content data relating to content; an output device for outputting the content; an acquisition device for acquiring response information indicating the user's response to notification information notified to the user by the output device; and an output mode changing device for changing the output mode of the content by the output device based on the notification information and the response information.
16. An information processing method comprising: a first acquisition step of acquiring response information indicating the user's response to notification information notified to the user from an output device; and a control step of changing the output mode of the output device based on the notification information and the response information.
17. An information processing program that causes a computer to perform: a first acquisition process for acquiring response information indicating the user's response to notification information notified to the user from an output device; and a control process for changing the output mode of the output device based on the notification information and the response information.
18. A terminal device comprising the information processing device described in claim 13.
