System

Abstract

To provide an interactive experience in which a viewer directly interacts with a character.SOLUTION: A system according to an embodiment includes a designation reception unit, an interaction unit, a selection reception unit, and a providing unit. The designation reception unit receives a designation of a character by a viewer in a content. The interaction unit executes interaction with the viewer by the character whose designation has been received by the designation reception unit. The selection reception unit receives selection of a story from the viewer. The providing unit provides the content as a content of the story whose selection has been received by the selection reception unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology had the drawback of not allowing viewers to directly interact with characters in the content, resulting in a lack of interactive experience.

[0005] The system according to the embodiment aims to provide an interactive experience by allowing a viewer to directly interact with characters in content. [Means for solving the problem]

[0006] The system according to the embodiment includes a designation receiving unit, a dialogue unit, a selection receiving unit, and a providing unit. The designation receiving unit receives designation of a character in content by a viewer. The dialogue unit executes a dialogue with the viewer using the character whose designation has been accepted by the designation receiving unit. The selection receiving unit accepts a story selection from the viewer. The providing unit provides the content as content of the story whose selection has been accepted by the selection receiving unit. Effect of the Invention

[0007] The system according to the embodiment allows the viewer to directly interact with the characters in the content, providing an interactive experience. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A system according to an embodiment of the present invention provides an interactive experience in which a viewer can directly interact with a character in an animation or video work. In this system, a viewer designates a character in the content through a designation reception unit, and a dialogue unit executes a dialogue with the character. The viewer selects a story through a selection reception unit, and a provision unit provides the content of the selected story. The dialogue unit recognizes the viewer's voice input and generates a voice output of the character. In addition, the dialogue unit recognizes the viewer's facial expressions and gestures, and changes the character's facial expressions and actions based on the selection. In addition, emotions are estimated from the viewer's voice and text, and the dialogue content is adjusted. The viewer can also select a voice actor for the character. The suggestion unit predicts the next story development based on the viewer's past selections, and the presentation unit presents story options based on historical backgrounds that interest the viewer. The dialogue unit executes dialogue between multiple characters selected by the viewer. For example, when a viewer is watching an animation work, the viewer can designate a specific character and start a dialogue with the character. The viewer asks a question to the character through a voice input, and the character responds with a voice output. In addition, the character's facial expressions and actions change according to the viewer's facial expressions and gestures, providing a more realistic dialogue experience. The dialogue content is adjusted based on the viewer's emotions, allowing dialogue to match the viewer's mood. Viewers can select the voice actor for the character and enjoy dialogue in their preferred voice. In addition, the next story development is predicted based on the viewer's past choices, and story options are presented based on the historical background that interests the viewer. This allows viewers to enjoy story development that matches their own interests and concerns. Viewers can also enjoy interactive experiences that allow them to directly interact with characters in animation and video works.

[0029] The interactive system according to the embodiment includes a designation receiving unit, a dialogue unit, a selection receiving unit, and a providing unit. The designation receiving unit receives designation of a character in the content by a viewer. The viewer can designate a character by, for example, clicking, a voice command, a gesture, or the like. The designation receiving unit receives the viewer's input and transmits the designated character to the dialogue unit. The dialogue unit executes a dialogue with the viewer by the character whose designation has been accepted by the designation receiving unit. The dialogue unit recognizes the viewer's voice input and generates a voice output of the character. For example, when the viewer says "Hello," the character responds with "Hello, what's wrong?". The dialogue unit also recognizes the viewer's facial expressions and gestures, and changes the character's facial expressions and actions based on the recognition. For example, when the viewer smiles, the character also smiles. Furthermore, the dialogue unit estimates emotions from the viewer's voice and text, and adjusts the dialogue content based on the estimated emotions. For example, when the viewer speaks in a sad voice, the character responds gently with "What's wrong?". The selection receiving unit accepts a selection of a story from the viewer. The viewer can select how the story will develop from the presented options. For example, if the viewer selects "Go Next," the story proceeds to the next scene. The providing unit provides content for the story whose selection has been accepted by the selection accepting unit. The providing unit provides content to the viewer based on the selected story. For example, if the viewer selects "Go on an adventure," the providing unit plays an adventure scene. In this way, the interactive system according to the embodiment can provide an interactive experience by allowing the viewer to interact with characters and select a story.

[0030] The designation reception unit accepts the designation of a character in the content by the viewer. The viewer can designate a character by, for example, clicking, using a voice command, using a gesture, or the like. Specifically, in the case of clicking, the viewer designates a character on the screen by clicking the mouse. In the case of using a voice command, the viewer issues a voice instruction such as "select character A," and the system analyzes the instruction using voice recognition technology. In the case of using a gesture, the viewer recognizes hand and body movements through a camera and designates a character with a specific gesture. The designation reception unit processes these inputs in real time and transmits information about the designated character to the dialogue unit. Furthermore, the designation reception unit can grasp the viewer's preferences and tendencies by recording the viewer's input history and learning past designation patterns. For example, if a specific character is frequently designated, customization is possible, such as displaying that character preferentially. As a result, the designation reception unit efficiently and accurately accepts the viewer's input, and realizes smooth information transmission to the dialogue unit.

[0031] The dialogue unit executes a dialogue with the viewer using the character whose designation has been accepted by the designation acceptance unit. The dialogue unit recognizes the voice input of the viewer and generates a voice output of the character. Specifically, when the viewer says "Hello," the character responds with "Hello, what's wrong?" The dialogue unit converts the viewer's remark into text using voice recognition technology, analyzes the text, and generates an appropriate response. Furthermore, the dialogue unit recognizes the viewer's facial expressions and gestures, and changes the character's facial expressions and movements based on the recognition. For example, when the viewer smiles, the character also smiles. For this, facial expression recognition technology and movement recognition technology using a camera are used. Furthermore, the dialogue unit estimates emotions from the viewer's voice and text, and adjusts the dialogue content based on the estimated emotions. For example, when the viewer speaks in a sad voice, the character responds gently with "What's wrong?". Natural language processing technology and machine learning algorithms are used for emotion estimation, and the viewer's emotional state can be grasped with high accuracy. As a result, the dialogue unit can realize a natural and emotional dialogue with the viewer, and provide an interactive experience.

[0032] The selection receiving unit receives a story selection from the viewer. The viewer can select a story development from the presented options. Specifically, the viewer selects the next story development by clicking an option displayed on the screen. For example, options such as "proceed to the next step" and "go on an adventure" are presented, and the viewer selects one from among them. The selection receiving unit receives the viewer's selection in real time and transmits the information to the providing unit. Furthermore, the selection receiving unit can grasp the viewer's preferences and tendencies by recording the viewer's selection history and learning the past selection patterns. For example, if a particular story development is frequently selected, customization is possible, such as presenting that development preferentially. In this way, the selection receiving unit efficiently and accurately receives the viewer's selection, and realizes smooth information transmission to the providing unit.

[0033] The providing unit provides the content of the story whose selection is accepted by the selection accepting unit. Specifically, when the viewer selects "Go on an adventure", the providing unit plays back the adventure scene. The providing unit provides the content to the viewer based on the selected story. For example, when the viewer selects "Go next", the story advances to the next scene. The providing unit uses high-speed data processing technology and streaming technology to provide appropriate content in real time according to the viewer's selection. Furthermore, the providing unit can collect viewer feedback and continuously improve the quality of the content and the method of providing it. For example, when a viewer gives a high rating to a particular scene, optimization such as providing that scene to more viewers is possible. This allows the providing unit to provide high-quality, interactive content to the viewer and improve the viewing experience.

[0034] The dialogue unit can recognize the viewer's voice input and generate a voice output for the character. For example, the dialogue unit obtains the viewer's voice input with a microphone and converts it into text data using a voice recognition algorithm. For example, when the viewer says "Hello," the dialogue unit converts the voice into text data and generates a response from the character. The dialogue unit also generates a voice output for the character using a voice synthesis technique. For example, the character responds with "Hello, what's the matter?". As a result, a dialogue is realized by the character responding based on the viewer's voice input. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, AI, or may be performed without using AI. For example, the dialogue unit can input the viewer's voice data to a generation AI and have the generation AI convert the voice data into text data.

[0035] The dialogue unit can recognize the viewer's facial expression and change the character's facial expression based on the viewer's facial expression. The dialogue unit, for example, photographs the viewer's facial expression using a camera and recognizes the facial expression using an image processing algorithm. For example, when the viewer smiles, the dialogue unit recognizes the facial expression and changes the character's facial expression to a smile. The dialogue unit also changes the character's facial expression using a facial expression database. For example, when the viewer shows a surprised expression, the character also shows a surprised expression. This allows the character's facial expression to change according to the viewer's facial expression, making it possible to have a more realistic dialogue. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, AI, or may be performed without using AI. For example, the dialogue unit can input the viewer's facial expression data to a generation AI and cause the generation AI to change the character's facial expression from the facial expression data.

[0036] The dialogue unit can recognize the gestures of the viewer and control the behavior of the character. The dialogue unit recognizes the gestures of the viewer using, for example, a motion sensor or a camera. For example, when the viewer waves his / her hand, the dialogue unit recognizes the gesture and the character also performs a waving action. The dialogue unit also controls the behavior of the character using a gesture database. For example, when the viewer points his / her finger, the character also performs a pointing action in the same direction. This allows the character's behavior to change according to the viewer's gesture, providing a more interactive experience. Some or all of the above-mentioned processing in the dialogue unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the dialogue unit can input the viewer's gesture data to the generation AI and cause the generation AI to control the character's behavior from the gesture data.

[0037] The dialogue unit may have a function that allows the viewer to select a voice actor of the character. For example, the dialogue unit may present the viewer with a choice of multiple voice actors, and the viewer may select a preferred voice actor. For example, when the viewer selects "voice actor A," the voice output of the character is generated in the voice of voice actor A. The dialogue unit may also generate the voice output of the character using the voice data of the voice actor selected by the viewer. For example, when the viewer selects "voice actor B," the voice output of the character is generated in the voice of voice actor B. This allows the viewer to select a voice actor for the character, providing a more personalized experience. Some or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit may input the viewer's selection data to the generation AI, and cause the generation AI to execute the voice data of the selected voice actor.

[0038] The present invention may include a suggestion unit that predicts the next story development based on the viewer's past selections and proposes the predicted story. The suggestion unit, for example, records the viewer's past selection history in a database and predicts the next story development using a prediction algorithm. For example, the suggestion unit analyzes the pattern of stories selected by the viewer in the past and proposes a story that is likely to be selected next. The suggestion unit also suggests a story that the viewer is likely to be interested in based on the viewer's selection history. For example, if the viewer has selected many action-related stories in the past, an action-related option is proposed as the next story. This makes it possible to continue to attract the viewer's interest by proposing the next story development based on the viewer's past selections. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, AI, or may be performed without using AI. For example, the suggestion unit may input the viewer's selection history data to the generation AI and cause the generation AI to execute a prediction of the next story development.

[0039] The presenting unit may include a presentation unit that presents story options based on historical backgrounds that interest the viewer. The presentation unit, for example, acquires historical backgrounds that interest the viewer from a database and presents story options. For example, if the viewer is interested in the history of medieval Europe, the presentation unit presents story options set in medieval Europe. The presentation unit also identifies historical backgrounds that the viewer is likely to be interested in based on the viewer's past viewing history and survey results. For example, if the viewer has watched many documentaries about the Warring States period in the past, the presentation unit presents story options set in the Warring States period. This makes it possible to continue to attract the viewer's interest by presenting story options based on the viewer's interests. A part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI, or may be performed without using AI. For example, the presentation unit may input the viewer's interest data to the generation AI and cause the generation AI to execute the presentation of story options.

[0040] The dialogue unit can execute dialogue between multiple characters selected by the viewer. For example, the dialogue unit executes dialogue between the characters when the viewer selects multiple characters. For example, when the viewer selects character A and character B, the dialogue unit generates a dialogue between character A and character B. The dialogue unit also designs a dialogue scenario based on the viewer's selection and realizes interaction between the characters. For example, when the viewer selects character C and character D, the dialogue unit generates a dialogue between character C and character D based on a specific scenario. As a result, a dialogue is executed between multiple characters selected by the viewer, enabling a more complex story development. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit inputs the viewer's selection data to the generation AI and causes the generation AI to execute a dialogue between multiple characters.

[0041] The designation reception unit can analyze the viewer's past character designation history and select the optimal designation method. The designation reception unit, for example, records the viewer's past character designation history in a database and selects the optimal designation method using an analysis algorithm. For example, the designation reception unit automatically displays characters frequently designated by the viewer in the past as candidates. In addition, the designation reception unit preferentially proposes designation methods (such as voice and text) used by the viewer in the past. Furthermore, the designation reception unit predicts and proposes characters to be designated in a specific time period from the viewer's past designation history. As a result, the optimal designation method is selected based on the viewer's past designation history, thereby making character designation more efficient. Part or all of the above-mentioned processing in the designation reception unit may be performed, for example, using AI or may be performed without using AI. For example, the designation reception unit can input the viewer's designation history data to a generation AI and cause the generation AI to select the optimal designation method.

[0042] The designation reception unit can filter based on the viewer's current field of interest when designating a character. The designation reception unit, for example, identifies the viewer's current field of interest from a survey result or past viewing history, and filters the characters. For example, characters in a genre in which the viewer is currently interested are preferentially displayed. Also, based on the viewer's recent viewing history, related characters are suggested. Furthermore, the viewer's social media activity is analyzed, and characters of high interest are filtered and displayed. As a result, characters are filtered based on the viewer's field of interest, and more appropriate characters are presented. A part or all of the above-mentioned processing in the designation reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the designation reception unit can input the viewer's interest data to the generation AI and cause the generation AI to execute character filtering.

[0043] When designating a character, the designation reception unit can preferentially designate a highly relevant character in consideration of the viewer's geographical location information. The designation reception unit, for example, acquires the viewer's geographical location information using GPS data or the like, and reflects it in the designation of the character. For example, when the viewer is in a specific area, characters related to that area are preferentially displayed. Also, when the viewer is traveling, characters related to the travel destination are suggested. Furthermore, based on the viewer's geographical location information, characters related to local culture and events are displayed. As a result, a character is designated based on the viewer's geographical location information, and a more relevant character is presented. A part or all of the above-mentioned processing in the designation reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the designation reception unit can input the viewer's geographical location data to the generation AI and cause the generation AI to designate a character.

[0044] The designation reception unit can analyze the viewer's social media activity and designate a related character when designating a character. For example, the designation reception unit obtains the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the designation reception unit preferentially displays characters that the viewer frequently mentions on social media. In addition, characters with many social media followers of the viewer are suggested. Furthermore, the content of the viewer's social media posts is analyzed and characters of high interest are displayed. As a result, a character is designated based on the viewer's social media activity, and a more related character is presented. A part or all of the above-mentioned processing in the designation reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the designation reception unit can input the viewer's social media data to the generation AI and cause the generation AI to designate a character.

[0045] The dialogue unit can generate optimal dialogue content by referring to the viewer's past dialogue history during dialogue. The dialogue unit, for example, records the viewer's past dialogue history in a database and generates optimal dialogue content using a generation algorithm. For example, a similar dialogue is generated based on dialogue content that the viewer liked in the past. Also, a dialogue with a specific character is preferentially generated from the viewer's past dialogue history. Furthermore, the viewer's past dialogue history is analyzed and a dialogue including a topic of interest is generated. As a result, optimal dialogue content is generated based on the viewer's past dialogue history, thereby making the dialogue more efficient. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI or may be performed without using AI. For example, the dialogue unit can input the viewer's dialogue history data to the generation AI and cause the generation AI to generate optimal dialogue content.

[0046] The dialogue unit can customize the dialogue content based on the viewer's current field of interest during the dialogue. For example, the dialogue unit identifies the viewer's current field of interest from a survey result or a past viewing history, and customizes the dialogue content. For example, the dialogue unit generates dialogue content based on a topic in which the viewer is currently interested. Also, based on the viewer's recent viewing history, related dialogue content is generated. Furthermore, the viewer's social media activity is analyzed, and dialogue content including topics of high interest is generated. As a result, the dialogue content is customized based on the viewer's field of interest, thereby realizing a more appropriate dialogue. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can input the viewer's interest data to the generation AI and cause the generation AI to customize the dialogue content.

[0047] The dialogue unit can generate optimal dialogue contents in consideration of the viewer's geographical location information during dialogue. The dialogue unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects it in the dialogue contents. For example, when the viewer is in a specific area, the dialogue unit generates dialogue contents including topics related to the area. Also, when the viewer is traveling, the dialogue unit generates dialogue contents including information related to the travel destination. Furthermore, based on the viewer's geographical location information, dialogue contents related to local culture and events are generated. As a result, dialogue contents are generated based on the viewer's geographical location information, thereby realizing more relevant dialogue. Part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI or may be performed without using AI. For example, the dialogue unit can input the viewer's geographical location data to the generation AI and cause the generation AI to generate the dialogue contents.

[0048] The dialogue unit can analyze the viewer's social media activity during the dialogue and generate related dialogue content. The dialogue unit, for example, acquires the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the dialogue unit generates dialogue content including topics that the viewer frequently mentions on social media. In addition, topics with many social media followers of the viewer are preferentially included in the dialogue content. Furthermore, the content posted on the viewer's social media is analyzed to generate dialogue content including topics of high interest. As a result, the dialogue content is generated based on the viewer's social media activity, thereby realizing a more relevant dialogue. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI or may be performed without using AI. For example, the dialogue unit can input the viewer's social media data to the generation AI and cause the generation AI to generate the dialogue content.

[0049] The selection receiving unit can analyze the viewer's past selection history and select the optimal selection method. The selection receiving unit, for example, records the viewer's past selection history in a database and selects the optimal selection method using an analysis algorithm. For example, the selection receiving unit automatically displays stories that the viewer has frequently selected in the past as candidates. In addition, the selection receiving unit preferentially suggests selection methods (audio, text, etc.) that the viewer has used in the past. Furthermore, the selection receiving unit predicts and suggests stories to be selected in a specific time period from the viewer's past selection history. As a result, the optimal selection method is selected based on the viewer's past selection history, thereby making the selection more efficient. Part or all of the above-mentioned processing in the selection receiving unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection receiving unit can input the viewer's selection history data to the generation AI and cause the generation AI to select the optimal selection method.

[0050] The selection reception unit can filter based on the viewer's current field of interest when selecting a story. The selection reception unit, for example, identifies the viewer's current field of interest from a survey result or past viewing history, and filters the stories. For example, stories in a genre in which the viewer is currently interested are preferentially displayed. Also, based on the viewer's recent viewing history, related stories are suggested. Furthermore, the viewer's social media activity is analyzed, and stories of high interest are filtered and displayed. As a result, stories are filtered based on the viewer's field of interest, and more appropriate stories are presented. A part or all of the above-mentioned processing in the selection reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection reception unit can input the viewer's interest data to the generation AI and have the generation AI execute filtering of the stories.

[0051] The selection receiving unit can, when selecting a story, preferentially select a highly relevant story in consideration of the viewer's geographical location information. The selection receiving unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects it in story selection. For example, when the viewer is in a specific area, stories related to that area are preferentially displayed. Also, when the viewer is traveling, stories related to the travel destination are suggested. Furthermore, stories related to local culture and events are displayed based on the viewer's geographical location information. As a result, stories are selected based on the viewer's geographical location information, and more relevant stories are presented. A part or all of the above-mentioned processing in the selection receiving unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection receiving unit can input the viewer's geographical location data to the generation AI and cause the generation AI to select a story.

[0052] The selection receiving unit can analyze the viewer's social media activity and select a related story when selecting a story. For example, the selection receiving unit obtains the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the selection receiving unit preferentially displays stories that the viewer frequently mentions on social media. In addition, the selection receiving unit suggests stories that have many social media followers of the viewer. Furthermore, the content of the viewer's social media posts is analyzed and stories of high interest are displayed. As a result, stories are selected based on the viewer's social media activity, and more relevant stories are presented. A part or all of the above-mentioned processing in the selection receiving unit may be performed using, for example, AI, or may be performed without using AI. For example, the selection receiving unit can input the viewer's social media data to the generation AI and cause the generation AI to select a story.

[0053] The providing unit can analyze the viewer's past content viewing history and select the optimal providing method. For example, the providing unit records the viewer's past content viewing history in a database and selects the optimal providing method using an analysis algorithm. For example, similar content is provided based on content that the viewer liked in the past. In addition, content of a specific genre is preferentially provided based on the viewer's past viewing history. Furthermore, the viewer's past viewing history is analyzed and content including topics of interest is provided. As a result, the optimal providing method is selected based on the viewer's past viewing history, thereby making the content provision more efficient. Part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the viewer's viewing history data to the generating AI and cause the generating AI to select the optimal providing method.

[0054] The providing unit can filter the content based on the viewer's current field of interest when providing the content. The providing unit, for example, identifies the viewer's current field of interest from a survey result or a past viewing history, and filters the content. For example, the providing unit preferentially provides content in a genre in which the viewer is currently interested. In addition, related content is provided based on the viewer's recent viewing history. Furthermore, the social media activity of the viewer is analyzed, and content of high interest is filtered and provided. As a result, more appropriate content is provided by filtering the content based on the viewer's field of interest. A part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the viewer's interest data to the generating AI and cause the generating AI to filter the content.

[0055] When providing content, the providing unit can provide highly relevant content preferentially in consideration of the viewer's geographical location information. The providing unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects the information in the content provided. For example, when the viewer is in a specific area, content related to the area is provided preferentially. Also, when the viewer is traveling, content related to the travel destination is provided. Furthermore, content related to local culture and events is provided based on the viewer's geographical location information. As a result, content is provided based on the viewer's geographical location information, so that more relevant content is provided. A part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the viewer's geographical location data to the generating AI and cause the generating AI to provide the content.

[0056] The providing unit can analyze the viewer's social media activity and provide related content when providing content. For example, the providing unit obtains the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the providing unit preferentially provides content that the viewer frequently mentions on social media. Also, the providing unit provides content that the viewer has many social media followers on. Furthermore, the providing unit analyzes the content posted on the viewer's social media and provides content that is of high interest to the viewer. As a result, content is provided based on the viewer's social media activity, so that more relevant content is provided. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the viewer's social media data to the generating AI and cause the generating AI to provide the content.

[0057] The suggestion unit can generate optimal suggested content by referring to the viewer's past selection history at the time of suggestion. The suggestion unit, for example, records the viewer's past selection history in a database and generates optimal suggested content using a generation algorithm. For example, the suggestion unit suggests similar stories based on stories that the viewer liked in the past. Also, stories of a specific genre are preferentially suggested from the viewer's past selection history. Furthermore, the viewer's past selection history is analyzed and a story including a topic of interest is suggested. As a result, optimal suggested content is generated based on the viewer's past selection history, making the suggestion more efficient. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input the viewer's selection history data to the generation AI and cause the generation AI to generate optimal suggested content.

[0058] The suggestion unit can customize the proposed content based on the viewer's current field of interest at the time of proposal. The suggestion unit, for example, identifies the viewer's current field of interest from a survey result or a past viewing history, and customizes the proposed content. For example, the suggestion unit generates the proposed content based on a topic in which the viewer is currently interested. Also, based on the viewer's recent viewing history, the suggestion unit generates related proposed content. Furthermore, the social media activity of the viewer is analyzed, and the proposed content including topics of high interest is generated. As a result, the proposed content is customized based on the viewer's field of interest, making it possible to make more appropriate proposals. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input the viewer's interest data to the generation AI, and cause the generation AI to customize the proposed content.

[0059] The suggestion unit can generate optimal proposal contents in consideration of the viewer's geographical location information at the time of proposal. The suggestion unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects it in the proposal contents. For example, when the viewer is in a specific area, the suggestion unit generates proposal contents including topics related to the area. Also, when the viewer is traveling, the suggestion unit generates proposal contents including information related to the travel destination. Furthermore, based on the viewer's geographical location information, the suggestion unit generates proposal contents related to local culture and events. As a result, the suggestion contents are generated based on the viewer's geographical location information, making it possible to make more relevant proposals. Part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input the viewer's geographical location data to the generation AI and cause the generation AI to execute generation of the proposal contents.

[0060] The suggestion unit can analyze the viewer's social media activity at the time of suggestion and generate relevant suggested content. The suggestion unit, for example, acquires the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the suggestion unit generates suggested content including topics that the viewer frequently mentions on social media. In addition, topics with many social media followers of the viewer are preferentially included in the suggested content. Furthermore, the content posted on the viewer's social media is analyzed and suggested content including topics of high interest is generated. As a result, the suggested content is generated based on the viewer's social media activity, making it possible to make more relevant suggestions. Some or all of the above-mentioned processing in the suggestion unit may be performed using, for example, AI, or may be performed without using AI. For example, the suggestion unit can input the viewer's social media data to the generation AI and cause the generation AI to generate the suggested content.

[0061] The presentation unit can generate optimal presentation content by referring to the viewer's past selection history at the time of presentation. The presentation unit, for example, records the viewer's past selection history in a database and generates optimal presentation content using a generation algorithm. For example, the presentation unit presents similar stories based on stories that the viewer liked in the past. In addition, stories of a specific genre are preferentially presented based on the viewer's past selection history. Furthermore, the viewer's past selection history is analyzed and stories including topics of interest are presented. As a result, optimal presentation content is generated based on the viewer's past selection history, making presentation more efficient. Part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI or may be performed without using AI. For example, the presentation unit can input the viewer's selection history data to the generation AI and cause the generation AI to generate optimal presentation content.

[0062] The presentation unit can customize the presentation content based on the viewer's current field of interest at the time of presentation. The presentation unit, for example, identifies the viewer's current field of interest from a survey result or a past viewing history, and customizes the presentation content. For example, the presentation content is generated based on a topic in which the viewer is currently interested. Also, based on the viewer's recent viewing history, related presentation content is generated. Furthermore, the viewer's social media activity is analyzed, and presentation content including topics of high interest is generated. As a result, the presentation content is customized based on the viewer's field of interest, making it possible to present more appropriately. Part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI, or may be performed without using AI. For example, the presentation unit can input the viewer's interest data to the generation AI, and cause the generation AI to customize the presentation content.

[0063] The presentation unit can generate optimal presentation contents in consideration of the viewer's geographical location information at the time of presentation. The presentation unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects the information in the presentation contents. For example, when the viewer is in a specific area, the presentation unit generates presentation contents including topics related to the area. Also, when the viewer is traveling, the presentation unit generates presentation contents including information related to the travel destination. Furthermore, based on the viewer's geographical location information, the presentation unit generates presentation contents related to local culture and events. As a result, the presentation contents are generated based on the viewer's geographical location information, making it possible to present the content with higher relevance. A part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI, or may be performed without using AI. For example, the presentation unit can input the viewer's geographical location data to the generation AI and cause the generation AI to generate the presentation contents.

[0064] The presentation unit can analyze the viewer's social media activity at the time of presentation and generate relevant presentation content. The presentation unit, for example, acquires the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the presentation unit generates presentation content including topics that the viewer frequently mentions on social media. In addition, topics with many social media followers of the viewer are preferentially included in the presentation content. Furthermore, the content posted on the viewer's social media is analyzed and presentation content including topics of high interest is generated. As a result, the presentation content is generated based on the viewer's social media activity, making it possible to present the content with higher relevance. A part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI or may be performed without using AI. For example, the presentation unit can input the viewer's social media data to the generation AI and cause the generation AI to generate the presentation content.

[0065] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0066] Based on the viewer's past selection history, the dialogue unit can learn the personality and speaking style of the character that the viewer likes, and customize the dialogue content. For example, if the character selected by the viewer in the past had a humorous personality, the dialogue unit will make the new character have a humorous personality as well. Also, if the character selected by the viewer in the past spoke politely, the dialogue unit will make the new character speak in a similar manner. Furthermore, if the character selected by the viewer in the past was knowledgeable about a particular topic, the dialogue unit will make the new character have knowledge about that topic. This allows dialogue to be tailored to the viewer's preferences.

[0067] When recognizing the viewer's voice input, the dialogue unit can learn the viewer's pronunciation and accent, making the character's responses more natural. For example, if the viewer uses a particular dialect, the dialogue unit learns that dialect and incorporates it into the character's responses. Also, if the viewer prefers a particular phrase, the dialogue unit learns that phrase and reflects it in the character's responses. Furthermore, if the viewer speaks at a particular tempo, the dialogue unit learns that tempo and the character's responses also use the same tempo. This makes the dialogue with the viewer more natural and familiar.

[0068] When recognizing the viewer's facial expression, the dialogue unit learns the viewer's facial features and can make the character's facial expression more similar to that of the viewer. For example, it learns how the corners of the mouth turn up and the eyes narrow when the viewer smiles, and reflects this in the character's smile. It also learns how the eyebrows turn up and the mouth opens when the viewer looks surprised, and reflects this in the character's surprised expression. It also learns how the eyes water and the mouth drops when the viewer looks sad, and reflects this in the character's sad expression. This strengthens the emotional connection with the viewer.

[0069] When recognizing the viewer's gestures, the dialogue unit learns the characteristics of the viewer's movements, allowing the character's movements to more closely resemble those of the viewer. For example, it can learn how the viewer moves their hand and the speed at which they wave, and reflect this in the way the character waves their hand. It can also learn the angle of the viewer's finger and wrist movement when they point, and reflect this in the way the character points. It can also learn the angle and movement of the neck when the viewer tilts their head, and reflect this in the way the character tilts their head. This makes interaction with the viewer more realistic and familiar.

[0070] When a viewer selects a voice actor for a character, the dialogue unit can refer to the viewer's past selection history and suggest the most suitable voice actor. For example, it can refer to the data of voice actors selected by the viewer in the past and suggest the same voice actor. It can also analyze the characteristics (tone of voice and speaking style) of voice actors selected by the viewer in the past and suggest similar voice actors. It can also analyze the genre (action, comedy, etc.) of voice actors selected by the viewer in the past and suggest voice actors in the same genre. This allows viewers to enjoy dialogue in their favorite voices.

[0071] When presenting story options based on historical backgrounds that interest the viewer, the presentation unit can refer to the viewer's past viewing history to present more relevant options. For example, if the viewer has viewed a lot of content related to the history of medieval Europe in the past, stories set in medieval Europe are presented preferentially. Also, if the viewer has viewed a lot of documentaries about the Warring States period in the past, stories set in the Warring States period are presented preferentially. Furthermore, if the viewer has viewed a lot of content related to the history of ancient Egypt in the past, stories set in ancient Egypt are presented preferentially. This makes it possible to select a story based on the viewer's interests.

[0072] The process flow of the first embodiment will be briefly described below.

[0073] Step 1: The specification reception unit receives the specification of a character in the content by the viewer. The viewer can specify a character by clicking, using a voice command, using a gesture, or by other methods. The specification reception unit receives the viewer's input and transmits the specified character to the dialogue unit. Step 2: The dialogue unit executes a dialogue with the viewer using the character whose designation has been accepted by the designation acceptance unit. The dialogue unit recognizes the viewer's voice input and generates a voice output for the character. For example, when the viewer says "Hello," the character responds with "Hello, what's the matter?" The dialogue unit also recognizes the viewer's facial expressions and gestures, and changes the character's facial expressions and actions based on those. Furthermore, the dialogue unit estimates emotions from the viewer's voice and text, and adjusts the content of the dialogue based on the estimated emotions. Step 3: The selection reception unit receives the story selection from the viewer. The viewer can choose the development of the story from the presented options. For example, if the viewer selects "Next", the story will proceed to the next scene. Step 4: The providing unit provides the content of the story whose selection has been accepted by the selection accepting unit. The providing unit provides the content to the viewer based on the selected story. For example, when the viewer selects "Go on an adventure", the providing unit plays an adventure scene.

[0074] (Example 2) A system according to an embodiment of the present invention provides an interactive experience in which a viewer can directly interact with a character in an animation or video work. In this system, a viewer designates a character in the content through a designation reception unit, and a dialogue unit executes a dialogue with the character. The viewer selects a story through a selection reception unit, and a provision unit provides the content of the selected story. The dialogue unit recognizes the viewer's voice input and generates a voice output of the character. In addition, the dialogue unit recognizes the viewer's facial expressions and gestures, and changes the character's facial expressions and actions based on the selection. In addition, emotions are estimated from the viewer's voice and text, and the dialogue content is adjusted. The viewer can also select a voice actor for the character. The suggestion unit predicts the next story development based on the viewer's past selections, and the presentation unit presents story options based on historical backgrounds that interest the viewer. The dialogue unit executes dialogue between multiple characters selected by the viewer. For example, when a viewer is watching an animation work, the viewer can designate a specific character and start a dialogue with the character. The viewer asks a question to the character through a voice input, and the character responds with a voice output. In addition, the character's facial expressions and actions change according to the viewer's facial expressions and gestures, providing a more realistic dialogue experience. The dialogue content is adjusted based on the viewer's emotions, allowing dialogue to match the viewer's mood. Viewers can select the voice actor for the character and enjoy dialogue in their preferred voice. In addition, the next story development is predicted based on the viewer's past choices, and story options are presented based on the historical background that interests the viewer. This allows viewers to enjoy story development that matches their own interests and concerns. Viewers can also enjoy interactive experiences that allow them to directly interact with characters in animation and video works.

[0075] The interactive system according to the embodiment includes a designation receiving unit, a dialogue unit, a selection receiving unit, and a providing unit. The designation receiving unit receives designation of a character in the content by a viewer. The viewer can designate a character by, for example, clicking, a voice command, a gesture, or the like. The designation receiving unit receives the viewer's input and transmits the designated character to the dialogue unit. The dialogue unit executes a dialogue with the viewer by the character whose designation has been accepted by the designation receiving unit. The dialogue unit recognizes the viewer's voice input and generates a voice output of the character. For example, when the viewer says "Hello," the character responds with "Hello, what's wrong?". The dialogue unit also recognizes the viewer's facial expressions and gestures, and changes the character's facial expressions and actions based on the recognition. For example, when the viewer smiles, the character also smiles. Furthermore, the dialogue unit estimates emotions from the viewer's voice and text, and adjusts the dialogue content based on the estimated emotions. For example, when the viewer speaks in a sad voice, the character responds gently with "What's wrong?". The selection receiving unit accepts a selection of a story from the viewer. The viewer can select how the story will develop from the presented options. For example, if the viewer selects "Go Next," the story proceeds to the next scene. The providing unit provides content for the story whose selection has been accepted by the selection accepting unit. The providing unit provides content to the viewer based on the selected story. For example, if the viewer selects "Go on an adventure," the providing unit plays an adventure scene. In this way, the interactive system according to the embodiment can provide an interactive experience by allowing the viewer to interact with characters and select a story.

[0076] The designation reception unit accepts the designation of a character in the content by the viewer. The viewer can designate a character by, for example, clicking, using a voice command, using a gesture, or the like. Specifically, in the case of clicking, the viewer designates a character on the screen by clicking the mouse. In the case of using a voice command, the viewer issues a voice instruction such as "select character A," and the system analyzes the instruction using voice recognition technology. In the case of using a gesture, the viewer recognizes hand and body movements through a camera and designates a character with a specific gesture. The designation reception unit processes these inputs in real time and transmits information about the designated character to the dialogue unit. Furthermore, the designation reception unit can grasp the viewer's preferences and tendencies by recording the viewer's input history and learning past designation patterns. For example, if a specific character is frequently designated, customization is possible, such as displaying that character preferentially. As a result, the designation reception unit efficiently and accurately accepts the viewer's input, and realizes smooth information transmission to the dialogue unit.

[0077] The dialogue unit executes a dialogue with the viewer using the character whose designation has been accepted by the designation acceptance unit. The dialogue unit recognizes the voice input of the viewer and generates a voice output of the character. Specifically, when the viewer says "Hello," the character responds with "Hello, what's wrong?" The dialogue unit converts the viewer's remark into text using voice recognition technology, analyzes the text, and generates an appropriate response. Furthermore, the dialogue unit recognizes the viewer's facial expressions and gestures, and changes the character's facial expressions and movements based on the recognition. For example, when the viewer smiles, the character also smiles. For this, facial expression recognition technology and movement recognition technology using a camera are used. Furthermore, the dialogue unit estimates emotions from the viewer's voice and text, and adjusts the dialogue content based on the estimated emotions. For example, when the viewer speaks in a sad voice, the character responds gently with "What's wrong?". Natural language processing technology and machine learning algorithms are used for emotion estimation, and the viewer's emotional state can be grasped with high accuracy. As a result, the dialogue unit can realize a natural and emotional dialogue with the viewer, and provide an interactive experience.

[0078] The selection receiving unit receives a story selection from the viewer. The viewer can select a story development from the presented options. Specifically, the viewer selects the next story development by clicking an option displayed on the screen. For example, options such as "proceed to the next step" and "go on an adventure" are presented, and the viewer selects one from among them. The selection receiving unit receives the viewer's selection in real time and transmits the information to the providing unit. Furthermore, the selection receiving unit can grasp the viewer's preferences and tendencies by recording the viewer's selection history and learning the past selection patterns. For example, if a particular story development is frequently selected, customization is possible, such as presenting that development preferentially. In this way, the selection receiving unit efficiently and accurately receives the viewer's selection, and realizes smooth information transmission to the providing unit.

[0079] The providing unit provides the content of the story whose selection is accepted by the selection accepting unit. Specifically, when the viewer selects "Go on an adventure", the providing unit plays back the adventure scene. The providing unit provides the content to the viewer based on the selected story. For example, when the viewer selects "Go next", the story advances to the next scene. The providing unit uses high-speed data processing technology and streaming technology to provide appropriate content in real time according to the viewer's selection. Furthermore, the providing unit can collect viewer feedback and continuously improve the quality of the content and the method of providing it. For example, when a viewer gives a high rating to a particular scene, optimization such as providing that scene to more viewers is possible. This allows the providing unit to provide high-quality, interactive content to the viewer and improve the viewing experience.

[0080] The dialogue unit can recognize the viewer's voice input and generate a voice output for the character. For example, the dialogue unit obtains the viewer's voice input with a microphone and converts it into text data using a voice recognition algorithm. For example, when the viewer says "Hello," the dialogue unit converts the voice into text data and generates a response from the character. The dialogue unit also generates a voice output for the character using a voice synthesis technique. For example, the character responds with "Hello, what's the matter?". As a result, a dialogue is realized by the character responding based on the viewer's voice input. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, AI, or may be performed without using AI. For example, the dialogue unit can input the viewer's voice data to a generation AI and have the generation AI convert the voice data into text data.

[0081] The dialogue unit can recognize the viewer's facial expression and change the character's facial expression based on the viewer's facial expression. The dialogue unit, for example, photographs the viewer's facial expression using a camera and recognizes the facial expression using an image processing algorithm. For example, when the viewer smiles, the dialogue unit recognizes the facial expression and changes the character's facial expression to a smile. The dialogue unit also changes the character's facial expression using a facial expression database. For example, when the viewer shows a surprised expression, the character also shows a surprised expression. This allows the character's facial expression to change according to the viewer's facial expression, making it possible to have a more realistic dialogue. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, AI, or may be performed without using AI. For example, the dialogue unit can input the viewer's facial expression data to a generation AI and cause the generation AI to change the character's facial expression from the facial expression data.

[0082] The dialogue unit can recognize the gestures of the viewer and control the behavior of the character. The dialogue unit recognizes the gestures of the viewer using, for example, a motion sensor or a camera. For example, when the viewer waves his / her hand, the dialogue unit recognizes the gesture and the character also performs a waving action. The dialogue unit also controls the behavior of the character using a gesture database. For example, when the viewer points his / her finger, the character also performs a pointing action in the same direction. This allows the character's behavior to change according to the viewer's gesture, providing a more interactive experience. Some or all of the above-mentioned processing in the dialogue unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the dialogue unit can input the viewer's gesture data to the generation AI and cause the generation AI to control the character's behavior from the gesture data.

[0083] The dialogue unit can estimate the emotion from the viewer's voice and text, and adjust the dialogue content based on the estimated emotion. The dialogue unit estimates the viewer's emotion using, for example, voice analysis technology or text analysis technology. For example, when the viewer speaks in a sad voice, the dialogue unit analyzes the voice and estimates the viewer's emotion as "sadness." Also, when the viewer inputs the text "I'm tired today," the dialogue unit analyzes the text and estimates the viewer's emotion as "fatigue." The dialogue unit adjusts the dialogue content based on the estimated emotion. For example, when the viewer speaks in a sad voice, the character responds gently with "What's wrong?" As a result, the dialogue content is adjusted according to the viewer's emotion, thereby realizing a more appropriate dialogue. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the dialogue unit can input the viewer's voice data and text data into the generation AI and have the generation AI estimate emotions.

[0084] The dialogue unit may have a function that allows the viewer to select a voice actor of the character. For example, the dialogue unit may present the viewer with a choice of multiple voice actors, and the viewer may select a preferred voice actor. For example, when the viewer selects "voice actor A," the voice output of the character is generated in the voice of voice actor A. The dialogue unit may also generate the voice output of the character using the voice data of the voice actor selected by the viewer. For example, when the viewer selects "voice actor B," the voice output of the character is generated in the voice of voice actor B. This allows the viewer to select a voice actor for the character, providing a more personalized experience. Some or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit may input the viewer's selection data to the generation AI, and cause the generation AI to execute the voice data of the selected voice actor.

[0085] The present invention may include a suggestion unit that predicts the next story development based on the viewer's past selections and proposes the predicted story. The suggestion unit, for example, records the viewer's past selection history in a database and predicts the next story development using a prediction algorithm. For example, the suggestion unit analyzes the pattern of stories selected by the viewer in the past and proposes a story that is likely to be selected next. The suggestion unit also suggests a story that the viewer is likely to be interested in based on the viewer's selection history. For example, if the viewer has selected many action-related stories in the past, an action-related option is proposed as the next story. This makes it possible to continue to attract the viewer's interest by proposing the next story development based on the viewer's past selections. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, AI, or may be performed without using AI. For example, the suggestion unit may input the viewer's selection history data to the generation AI and cause the generation AI to execute a prediction of the next story development.

[0086] The presenting unit may include a presentation unit that presents story options based on historical backgrounds that interest the viewer. The presentation unit, for example, acquires historical backgrounds that interest the viewer from a database and presents story options. For example, if the viewer is interested in the history of medieval Europe, the presentation unit presents story options set in medieval Europe. The presentation unit also identifies historical backgrounds that the viewer is likely to be interested in based on the viewer's past viewing history and survey results. For example, if the viewer has watched many documentaries about the Warring States period in the past, the presentation unit presents story options set in the Warring States period. This makes it possible to continue to attract the viewer's interest by presenting story options based on the viewer's interests. A part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI, or may be performed without using AI. For example, the presentation unit may input the viewer's interest data to the generation AI and cause the generation AI to execute the presentation of story options.

[0087] The dialogue unit can execute dialogue between multiple characters selected by the viewer. For example, the dialogue unit executes dialogue between the characters when the viewer selects multiple characters. For example, when the viewer selects character A and character B, the dialogue unit generates a dialogue between character A and character B. The dialogue unit also designs a dialogue scenario based on the viewer's selection and realizes interaction between the characters. For example, when the viewer selects character C and character D, the dialogue unit generates a dialogue between character C and character D based on a specific scenario. As a result, a dialogue is executed between multiple characters selected by the viewer, enabling a more complex story development. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit inputs the viewer's selection data to the generation AI and causes the generation AI to execute a dialogue between multiple characters.

[0088] The designation reception unit can estimate the viewer's emotion and adjust the character designation method based on the estimated viewer's emotion. The designation reception unit estimates the viewer's emotion using, for example, voice analysis or text analysis, and adjusts the character designation method. For example, when the viewer is excited, the designation reception unit provides an interface that allows the viewer to quickly designate a character. Furthermore, when the viewer is relaxed, the designation reception unit displays detailed character information and increases the number of options. Furthermore, when the viewer is stressed, the designation reception unit provides a simple interface and minimizes the character designation procedure. This allows the character designation method to be adjusted according to the viewer's emotion, thereby enabling more appropriate character designation. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the designation reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the designation reception unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the character designation method.

[0089] The designation reception unit can analyze the viewer's past character designation history and select the optimal designation method. The designation reception unit, for example, records the viewer's past character designation history in a database and selects the optimal designation method using an analysis algorithm. For example, the designation reception unit automatically displays characters frequently designated by the viewer in the past as candidates. In addition, the designation reception unit preferentially proposes designation methods (such as voice and text) used by the viewer in the past. Furthermore, the designation reception unit predicts and proposes characters to be designated in a specific time period from the viewer's past designation history. As a result, the optimal designation method is selected based on the viewer's past designation history, thereby making character designation more efficient. Part or all of the above-mentioned processing in the designation reception unit may be performed, for example, using AI or may be performed without using AI. For example, the designation reception unit can input the viewer's designation history data to a generation AI and cause the generation AI to select the optimal designation method.

[0090] The designation reception unit can filter based on the viewer's current field of interest when designating a character. The designation reception unit, for example, identifies the viewer's current field of interest from a survey result or past viewing history, and filters the characters. For example, characters in a genre in which the viewer is currently interested are preferentially displayed. Also, based on the viewer's recent viewing history, related characters are suggested. Furthermore, the viewer's social media activity is analyzed, and characters of high interest are filtered and displayed. As a result, characters are filtered based on the viewer's field of interest, and more appropriate characters are presented. A part or all of the above-mentioned processing in the designation reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the designation reception unit can input the viewer's interest data to the generation AI and cause the generation AI to execute character filtering.

[0091] The designation reception unit can estimate the viewer's emotion and determine the priority order of the characters to be designated based on the estimated viewer's emotion. The designation reception unit, for example, estimates the viewer's emotion using voice analysis or text analysis and determines the priority order of the characters. For example, when the viewer is excited, action characters are preferentially displayed. Furthermore, when the viewer is relaxed, soothing characters are preferentially displayed. Furthermore, when the viewer is stressed, comedy characters are preferentially displayed. In this way, the priority order of the characters is determined according to the viewer's emotion, and more appropriate characters are presented. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the designation reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the designation reception unit can input the viewer's emotion data to the generation AI and cause the generation AI to execute the determination of the priority order of the characters.

[0092] When designating a character, the designation reception unit can preferentially designate a highly relevant character in consideration of the viewer's geographical location information. The designation reception unit, for example, acquires the viewer's geographical location information using GPS data or the like, and reflects it in the designation of the character. For example, when the viewer is in a specific area, characters related to that area are preferentially displayed. Also, when the viewer is traveling, characters related to the travel destination are suggested. Furthermore, based on the viewer's geographical location information, characters related to local culture and events are displayed. As a result, a character is designated based on the viewer's geographical location information, and a more relevant character is presented. A part or all of the above-mentioned processing in the designation reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the designation reception unit can input the viewer's geographical location data to the generation AI and cause the generation AI to designate a character.

[0093] The designation reception unit can analyze the viewer's social media activity and designate a related character when designating a character. For example, the designation reception unit obtains the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the designation reception unit preferentially displays characters that the viewer frequently mentions on social media. In addition, characters with many social media followers of the viewer are suggested. Furthermore, the content of the viewer's social media posts is analyzed and characters of high interest are displayed. As a result, a character is designated based on the viewer's social media activity, and a more related character is presented. A part or all of the above-mentioned processing in the designation reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the designation reception unit can input the viewer's social media data to the generation AI and cause the generation AI to designate a character.

[0094] The dialogue unit can estimate the viewer's emotion and adjust the dialogue expression method based on the estimated viewer's emotion. The dialogue unit, for example, estimates the viewer's emotion using voice analysis or text analysis and adjusts the dialogue expression method. For example, if the viewer is excited, the dialogue tempo is increased and energetic expressions are used. Also, if the viewer is relaxed, the dialogue tempo is decreased and calm expressions are used. Furthermore, if the viewer is stressed, the dialogue tempo is decreased and expressions that give a sense of security are used. In this way, the dialogue expression method is adjusted according to the viewer's emotion, thereby realizing a more appropriate dialogue. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the dialogue unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the dialogue expression method.

[0095] The dialogue unit can generate optimal dialogue content by referring to the viewer's past dialogue history during dialogue. The dialogue unit, for example, records the viewer's past dialogue history in a database and generates optimal dialogue content using a generation algorithm. For example, a similar dialogue is generated based on dialogue content that the viewer liked in the past. Also, a dialogue with a specific character is preferentially generated from the viewer's past dialogue history. Furthermore, the viewer's past dialogue history is analyzed and a dialogue including a topic of interest is generated. As a result, optimal dialogue content is generated based on the viewer's past dialogue history, thereby making the dialogue more efficient. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI or may be performed without using AI. For example, the dialogue unit can input the viewer's dialogue history data to the generation AI and cause the generation AI to generate optimal dialogue content.

[0096] The dialogue unit can customize the dialogue content based on the viewer's current field of interest during the dialogue. For example, the dialogue unit identifies the viewer's current field of interest from a survey result or a past viewing history, and customizes the dialogue content. For example, the dialogue unit generates dialogue content based on a topic in which the viewer is currently interested. Also, based on the viewer's recent viewing history, related dialogue content is generated. Furthermore, the viewer's social media activity is analyzed, and dialogue content including topics of high interest is generated. As a result, the dialogue content is customized based on the viewer's field of interest, thereby realizing a more appropriate dialogue. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can input the viewer's interest data to the generation AI and cause the generation AI to customize the dialogue content.

[0097] The dialogue unit can estimate the viewer's emotion and adjust the length of the dialogue based on the estimated viewer's emotion. The dialogue unit, for example, estimates the viewer's emotion using voice analysis or text analysis and adjusts the length of the dialogue. For example, if the viewer is excited, the dialogue length is shortened and the tempo is increased. Also, if the viewer is relaxed, the dialogue length is increased and includes detailed explanations. Furthermore, if the viewer is stressed, the dialogue length is shortened and the content is made to focus on the main points. As a result, the dialogue length is adjusted according to the viewer's emotion, thereby realizing a more appropriate dialogue. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the dialogue unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the length of the dialogue.

[0098] The dialogue unit can generate optimal dialogue contents in consideration of the viewer's geographical location information during dialogue. The dialogue unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects it in the dialogue contents. For example, when the viewer is in a specific area, the dialogue unit generates dialogue contents including topics related to the area. Also, when the viewer is traveling, the dialogue unit generates dialogue contents including information related to the travel destination. Furthermore, based on the viewer's geographical location information, dialogue contents related to local culture and events are generated. As a result, dialogue contents are generated based on the viewer's geographical location information, thereby realizing more relevant dialogue. Part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI or may be performed without using AI. For example, the dialogue unit can input the viewer's geographical location data to the generation AI and cause the generation AI to generate the dialogue contents.

[0099] The dialogue unit can analyze the viewer's social media activity during the dialogue and generate related dialogue content. The dialogue unit, for example, acquires the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the dialogue unit generates dialogue content including topics that the viewer frequently mentions on social media. In addition, topics with many social media followers of the viewer are preferentially included in the dialogue content. Furthermore, the content posted on the viewer's social media is analyzed to generate dialogue content including topics of high interest. As a result, the dialogue content is generated based on the viewer's social media activity, thereby realizing a more relevant dialogue. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI or may be performed without using AI. For example, the dialogue unit can input the viewer's social media data to the generation AI and cause the generation AI to generate the dialogue content.

[0100] The selection receiving unit can estimate the viewer's emotion and adjust the selection receiving method based on the estimated viewer's emotion. The selection receiving unit, for example, estimates the viewer's emotion using voice analysis or text analysis and adjusts the selection receiving method. For example, when the viewer is excited, options are displayed quickly to simplify the selection procedure. Also, when the viewer is relaxed, detailed options are displayed to increase the options. Furthermore, when the viewer is stressed, a simple interface is provided to minimize the selection procedure. This allows a more appropriate selection by adjusting the selection receiving method according to the viewer's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the selection receiving unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the selection receiving unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the selection receiving method.

[0101] The selection receiving unit can analyze the viewer's past selection history and select the optimal selection method. The selection receiving unit, for example, records the viewer's past selection history in a database and selects the optimal selection method using an analysis algorithm. For example, the selection receiving unit automatically displays stories that the viewer has frequently selected in the past as candidates. In addition, the selection receiving unit preferentially suggests selection methods (audio, text, etc.) that the viewer has used in the past. Furthermore, the selection receiving unit predicts and suggests stories to be selected in a specific time period from the viewer's past selection history. As a result, the optimal selection method is selected based on the viewer's past selection history, thereby making the selection more efficient. Part or all of the above-mentioned processing in the selection receiving unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection receiving unit can input the viewer's selection history data to the generation AI and cause the generation AI to select the optimal selection method.

[0102] The selection reception unit can filter based on the viewer's current field of interest when selecting a story. The selection reception unit, for example, identifies the viewer's current field of interest from a survey result or past viewing history, and filters the stories. For example, stories in a genre in which the viewer is currently interested are preferentially displayed. Also, based on the viewer's recent viewing history, related stories are suggested. Furthermore, the viewer's social media activity is analyzed, and stories of high interest are filtered and displayed. As a result, stories are filtered based on the viewer's field of interest, and more appropriate stories are presented. A part or all of the above-mentioned processing in the selection reception unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection reception unit can input the viewer's interest data to the generation AI and have the generation AI execute filtering of the stories.

[0103] The selection receiving unit can estimate the viewer's emotion and determine the priority order of stories to be selected based on the estimated viewer's emotion. The selection receiving unit, for example, estimates the viewer's emotion using voice analysis or text analysis and determines the priority order of stories. For example, if the viewer is excited, action stories are preferentially displayed. Also, if the viewer is relaxed, healing stories are preferentially displayed. Furthermore, if the viewer is stressed, comedy stories are preferentially displayed. As a result, the priority order of stories is determined according to the viewer's emotion, and more appropriate stories are presented. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the selection receiving unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the selection receiving unit can input the viewer's emotion data to the generation AI and cause the generation AI to execute the determination of the priority order of stories.

[0104] The selection receiving unit can, when selecting a story, preferentially select a highly relevant story in consideration of the viewer's geographical location information. The selection receiving unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects it in story selection. For example, when the viewer is in a specific area, stories related to that area are preferentially displayed. Also, when the viewer is traveling, stories related to the travel destination are suggested. Furthermore, stories related to local culture and events are displayed based on the viewer's geographical location information. As a result, stories are selected based on the viewer's geographical location information, and more relevant stories are presented. A part or all of the above-mentioned processing in the selection receiving unit may be performed, for example, using AI, or may be performed without using AI. For example, the selection receiving unit can input the viewer's geographical location data to the generation AI and cause the generation AI to select a story.

[0105] The selection receiving unit can analyze the viewer's social media activity and select a related story when selecting a story. For example, the selection receiving unit obtains the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the selection receiving unit preferentially displays stories that the viewer frequently mentions on social media. In addition, the selection receiving unit suggests stories that have many social media followers of the viewer. Furthermore, the content of the viewer's social media posts is analyzed and stories of high interest are displayed. As a result, stories are selected based on the viewer's social media activity, and more relevant stories are presented. A part or all of the above-mentioned processing in the selection receiving unit may be performed using, for example, AI, or may be performed without using AI. For example, the selection receiving unit can input the viewer's social media data to the generation AI and cause the generation AI to select a story.

[0106] The providing unit can estimate the viewer's emotion and adjust the content providing method based on the estimated viewer's emotion. The providing unit, for example, estimates the viewer's emotion using voice analysis or text analysis and adjusts the content providing method. For example, if the viewer is excited, fast-paced content is provided. Also, if the viewer is relaxed, calm content is provided. Furthermore, if the viewer is stressed, relaxing content is provided. In this way, the content providing method is adjusted according to the viewer's emotion, and more appropriate content is provided. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the content providing method.

[0107] The providing unit can analyze the viewer's past content viewing history and select the optimal providing method. For example, the providing unit records the viewer's past content viewing history in a database and selects the optimal providing method using an analysis algorithm. For example, similar content is provided based on content that the viewer liked in the past. In addition, content of a specific genre is preferentially provided based on the viewer's past viewing history. Furthermore, the viewer's past viewing history is analyzed and content including topics of interest is provided. As a result, the optimal providing method is selected based on the viewer's past viewing history, thereby making the content provision more efficient. Part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the viewer's viewing history data to the generating AI and cause the generating AI to select the optimal providing method.

[0108] The providing unit can filter the content based on the viewer's current field of interest when providing the content. The providing unit, for example, identifies the viewer's current field of interest from a survey result or a past viewing history, and filters the content. For example, the providing unit preferentially provides content in a genre in which the viewer is currently interested. In addition, related content is provided based on the viewer's recent viewing history. Furthermore, the social media activity of the viewer is analyzed, and content of high interest is filtered and provided. As a result, more appropriate content is provided by filtering the content based on the viewer's field of interest. A part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the viewer's interest data to the generating AI and cause the generating AI to filter the content.

[0109] The providing unit can estimate the viewer's emotion and determine the priority order of the content to be provided based on the estimated viewer's emotion. The providing unit, for example, estimates the viewer's emotion using voice analysis or text analysis and determines the priority order of the content. For example, if the viewer is excited, action content is provided preferentially. Also, if the viewer is relaxed, healing content is provided preferentially. Furthermore, if the viewer is stressed, comedy content is provided preferentially. As a result, the priority order of the content is determined according to the viewer's emotion, and more appropriate content is provided. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the viewer's emotion data to the generation AI and cause the generation AI to execute the determination of the priority order of the content.

[0110] When providing content, the providing unit can provide highly relevant content preferentially in consideration of the viewer's geographical location information. The providing unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects the information in the content provided. For example, when the viewer is in a specific area, content related to the area is provided preferentially. Also, when the viewer is traveling, content related to the travel destination is provided. Furthermore, content related to local culture and events is provided based on the viewer's geographical location information. As a result, content is provided based on the viewer's geographical location information, so that more relevant content is provided. A part or all of the above-mentioned processing in the providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the providing unit can input the viewer's geographical location data to the generating AI and cause the generating AI to provide the content.

[0111] The providing unit can analyze the viewer's social media activity and provide related content when providing content. For example, the providing unit obtains the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the providing unit preferentially provides content that the viewer frequently mentions on social media. Also, the providing unit provides content that the viewer has many social media followers on. Furthermore, the providing unit analyzes the content posted on the viewer's social media and provides content that is of high interest to the viewer. As a result, content is provided based on the viewer's social media activity, so that more relevant content is provided. A part or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the viewer's social media data to the generating AI and cause the generating AI to provide the content.

[0112] The suggestion unit can estimate the viewer's emotion and adjust the expression method of the proposal based on the estimated viewer's emotion. The suggestion unit, for example, estimates the viewer's emotion using voice analysis or text analysis and adjusts the expression method of the proposal. For example, if the viewer is excited, the suggestion is made using an energetic expression. If the viewer is relaxed, the suggestion is made using a calm expression. If the viewer is stressed, the suggestion is made using an expression that gives a sense of security. This allows the expression method of the proposal to be adjusted according to the viewer's emotion, making it possible to make a more appropriate proposal. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the expression method of the proposal.

[0113] The suggestion unit can generate optimal suggested content by referring to the viewer's past selection history at the time of suggestion. The suggestion unit, for example, records the viewer's past selection history in a database and generates optimal suggested content using a generation algorithm. For example, the suggestion unit suggests similar stories based on stories that the viewer liked in the past. Also, stories of a specific genre are preferentially suggested from the viewer's past selection history. Furthermore, the viewer's past selection history is analyzed and a story including a topic of interest is suggested. As a result, optimal suggested content is generated based on the viewer's past selection history, making the suggestion more efficient. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input the viewer's selection history data to the generation AI and cause the generation AI to generate optimal suggested content.

[0114] The suggestion unit can customize the proposed content based on the viewer's current field of interest at the time of proposal. The suggestion unit, for example, identifies the viewer's current field of interest from a survey result or a past viewing history, and customizes the proposed content. For example, the suggestion unit generates the proposed content based on a topic in which the viewer is currently interested. Also, based on the viewer's recent viewing history, the suggestion unit generates related proposed content. Furthermore, the social media activity of the viewer is analyzed, and the proposed content including topics of high interest is generated. As a result, the proposed content is customized based on the viewer's field of interest, making it possible to make more appropriate proposals. A part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI, or may be performed without using AI. For example, the suggestion unit can input the viewer's interest data to the generation AI, and cause the generation AI to customize the proposed content.

[0115] The suggestion unit can estimate the viewer's emotion and adjust the length of the proposal based on the estimated viewer's emotion. The suggestion unit, for example, estimates the viewer's emotion using voice analysis or text analysis and adjusts the length of the proposal. For example, if the viewer is excited, the length of the proposal is shortened and the tempo is increased. Also, if the viewer is relaxed, the length of the proposal is increased and includes detailed explanations. Furthermore, if the viewer is stressed, the length of the proposal is shortened and the content is made to focus on the main points. As a result, the length of the proposal is adjusted according to the viewer's emotion, enabling more appropriate proposals. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the suggestion unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the suggestion unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the length of the proposal.

[0116] The suggestion unit can generate optimal proposal contents in consideration of the viewer's geographical location information at the time of proposal. The suggestion unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects it in the proposal contents. For example, when the viewer is in a specific area, the suggestion unit generates proposal contents including topics related to the area. Also, when the viewer is traveling, the suggestion unit generates proposal contents including information related to the travel destination. Furthermore, based on the viewer's geographical location information, the suggestion unit generates proposal contents related to local culture and events. As a result, the suggestion contents are generated based on the viewer's geographical location information, making it possible to make more relevant proposals. Part or all of the above-mentioned processing in the suggestion unit may be performed, for example, using AI or may be performed without using AI. For example, the suggestion unit can input the viewer's geographical location data to the generation AI and cause the generation AI to execute generation of the proposal contents.

[0117] The suggestion unit can analyze the viewer's social media activity at the time of suggestion and generate relevant suggested content. The suggestion unit, for example, acquires the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the suggestion unit generates suggested content including topics that the viewer frequently mentions on social media. In addition, topics with many social media followers of the viewer are preferentially included in the suggested content. Furthermore, the content posted on the viewer's social media is analyzed and suggested content including topics of high interest is generated. As a result, the suggested content is generated based on the viewer's social media activity, making it possible to make more relevant suggestions. Some or all of the above-mentioned processing in the suggestion unit may be performed using, for example, AI, or may be performed without using AI. For example, the suggestion unit can input the viewer's social media data to the generation AI and cause the generation AI to generate the suggested content.

[0118] The presentation unit can estimate the viewer's emotion and adjust the presentation expression method based on the estimated viewer's emotion. The presentation unit, for example, estimates the viewer's emotion using voice analysis or text analysis and adjusts the presentation expression method. For example, if the viewer is excited, the presentation is made using an energetic expression. If the viewer is relaxed, the presentation is made using a calm expression. If the viewer is stressed, the presentation is made using an expression that gives a sense of security. This allows the presentation to be more appropriate by adjusting the presentation expression method according to the viewer's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the presentation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the presentation unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the presentation expression method.

[0119] The presentation unit can generate optimal presentation content by referring to the viewer's past selection history at the time of presentation. The presentation unit, for example, records the viewer's past selection history in a database and generates optimal presentation content using a generation algorithm. For example, the presentation unit presents similar stories based on stories that the viewer liked in the past. In addition, stories of a specific genre are preferentially presented based on the viewer's past selection history. Furthermore, the viewer's past selection history is analyzed and stories including topics of interest are presented. As a result, optimal presentation content is generated based on the viewer's past selection history, making presentation more efficient. Part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI or may be performed without using AI. For example, the presentation unit can input the viewer's selection history data to the generation AI and cause the generation AI to generate optimal presentation content.

[0120] The presentation unit can customize the presentation content based on the viewer's current field of interest at the time of presentation. The presentation unit, for example, identifies the viewer's current field of interest from a survey result or a past viewing history, and customizes the presentation content. For example, the presentation content is generated based on a topic in which the viewer is currently interested. Also, based on the viewer's recent viewing history, related presentation content is generated. Furthermore, the viewer's social media activity is analyzed, and presentation content including topics of high interest is generated. As a result, the presentation content is customized based on the viewer's field of interest, making it possible to present more appropriately. Part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI, or may be performed without using AI. For example, the presentation unit can input the viewer's interest data to the generation AI, and cause the generation AI to customize the presentation content.

[0121] The presentation unit can estimate the viewer's emotion and adjust the presentation length based on the estimated viewer's emotion. The presentation unit, for example, estimates the viewer's emotion using voice analysis or text analysis and adjusts the presentation length. For example, if the viewer is excited, the presentation length is shortened and the tempo is increased. Also, if the viewer is relaxed, the presentation length is increased and includes detailed explanations. Furthermore, if the viewer is stressed, the presentation length is shortened and the content is made to focus on the main points. This allows the presentation length to be adjusted according to the viewer's emotion, making it possible to present more appropriately. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the presentation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the presentation unit can input the viewer's emotion data to the generation AI and cause the generation AI to adjust the presentation length.

[0122] The presentation unit can generate optimal presentation contents in consideration of the viewer's geographical location information at the time of presentation. The presentation unit, for example, acquires the viewer's geographical location information using GPS data or the like and reflects the information in the presentation contents. For example, when the viewer is in a specific area, the presentation unit generates presentation contents including topics related to the area. Also, when the viewer is traveling, the presentation unit generates presentation contents including information related to the travel destination. Furthermore, based on the viewer's geographical location information, the presentation unit generates presentation contents related to local culture and events. As a result, the presentation contents are generated based on the viewer's geographical location information, making it possible to present the content with higher relevance. A part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI, or may be performed without using AI. For example, the presentation unit can input the viewer's geographical location data to the generation AI and cause the generation AI to generate the presentation contents.

[0123] The presentation unit can analyze the viewer's social media activity at the time of presentation and generate relevant presentation content. The presentation unit, for example, acquires the viewer's social media activity using an API and analyzes it using a data analysis algorithm. For example, the presentation unit generates presentation content including topics that the viewer frequently mentions on social media. In addition, topics with many social media followers of the viewer are preferentially included in the presentation content. Furthermore, the content posted on the viewer's social media is analyzed and presentation content including topics of high interest is generated. As a result, the presentation content is generated based on the viewer's social media activity, making it possible to present the content with higher relevance. A part or all of the above-mentioned processing in the presentation unit may be performed, for example, using AI or may be performed without using AI. For example, the presentation unit can input the viewer's social media data to the generation AI and cause the generation AI to generate the presentation content.

[0124] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0125] Based on the viewer's past selection history, the dialogue unit can learn the personality and speaking style of the character that the viewer likes, and customize the dialogue content. For example, if the character selected by the viewer in the past had a humorous personality, the dialogue unit will make the new character have a humorous personality as well. Also, if the character selected by the viewer in the past spoke politely, the dialogue unit will make the new character speak in a similar manner. Furthermore, if the character selected by the viewer in the past was knowledgeable about a particular topic, the dialogue unit will make the new character have knowledge about that topic. This allows dialogue to be tailored to the viewer's preferences.

[0126] When recognizing the viewer's voice input, the dialogue unit can learn the viewer's pronunciation and accent, making the character's responses more natural. For example, if the viewer uses a particular dialect, the dialogue unit learns that dialect and incorporates it into the character's responses. Also, if the viewer prefers a particular phrase, the dialogue unit learns that phrase and reflects it in the character's responses. Furthermore, if the viewer speaks at a particular tempo, the dialogue unit learns that tempo and the character's responses also use the same tempo. This makes the dialogue with the viewer more natural and familiar.

[0127] When recognizing the viewer's facial expression, the dialogue unit learns the viewer's facial features and can make the character's facial expression more similar to that of the viewer. For example, it learns how the corners of the mouth turn up and the eyes narrow when the viewer smiles, and reflects this in the character's smile. It also learns how the eyebrows turn up and the mouth opens when the viewer looks surprised, and reflects this in the character's surprised expression. It also learns how the eyes water and the mouth drops when the viewer looks sad, and reflects this in the character's sad expression. This strengthens the emotional connection with the viewer.

[0128] When recognizing the viewer's gestures, the dialogue unit learns the characteristics of the viewer's movements, allowing the character's movements to more closely resemble those of the viewer. For example, it can learn how the viewer moves their hand and the speed at which they wave, and reflect this in the way the character waves their hand. It can also learn the angle of the viewer's finger and wrist movement when they point, and reflect this in the way the character points. It can also learn the angle and movement of the neck when the viewer tilts their head, and reflect this in the way the character tilts their head. This makes interaction with the viewer more realistic and familiar.

[0129] When estimating emotions from the viewer's voice and text, the dialogue unit can perform more accurate emotion estimation by referring to the viewer's past emotion history. For example, the dialogue unit refers to voice data when the viewer spoke in a sad voice in the past and compares it with the current voice data. Also, the dialogue unit refers to text data in which the viewer previously input "I'm tired" and compares it with the current text data. Furthermore, the dialogue unit refers to voice and text data when the viewer previously expressed anger and compares it with the current data. This allows the dialogue content to be more appropriately adjusted based on the viewer's emotions.

[0130] When a viewer selects a voice actor for a character, the dialogue unit can refer to the viewer's past selection history and suggest the most suitable voice actor. For example, it can refer to the data of voice actors selected by the viewer in the past and suggest the same voice actor. It can also analyze the characteristics (tone of voice and speaking style) of voice actors selected by the viewer in the past and suggest similar voice actors. It can also analyze the genre (action, comedy, etc.) of voice actors selected by the viewer in the past and suggest voice actors in the same genre. This allows viewers to enjoy dialogue in their favorite voices.

[0131] The suggestion unit can take into account the viewer's current emotional state when predicting the next story development based on the viewer's past selection history. For example, if the viewer is currently excited, a story with many action scenes is suggested. If the viewer is currently relaxed, a story with many calm scenes is suggested. If the viewer is currently stressed, a story with many comedy scenes is suggested. In this way, a story development according to the viewer's emotional state is suggested.

[0132] When presenting story options based on historical backgrounds that interest the viewer, the presentation unit can refer to the viewer's past viewing history to present more relevant options. For example, if the viewer has viewed a lot of content related to the history of medieval Europe in the past, stories set in medieval Europe are presented preferentially. Also, if the viewer has viewed a lot of documentaries about the Warring States period in the past, stories set in the Warring States period are presented preferentially. Furthermore, if the viewer has viewed a lot of content related to the history of ancient Egypt in the past, stories set in ancient Egypt are presented preferentially. This makes it possible to select a story based on the viewer's interests.

[0133] When the dialogue unit executes dialogue between multiple characters selected by the viewer, it can adjust the tone and content of the dialogue in consideration of the emotional state of the viewer. For example, if the viewer is currently excited, the dialogue between the characters can be executed in an energetic tone. If the viewer is currently relaxed, the dialogue between the characters can be executed in a calm tone. Furthermore, if the viewer is currently stressed, the dialogue between the characters can be executed in a reassuring manner. In this way, dialogue according to the emotional state of the viewer can be realized.

[0134] The designation reception unit can estimate the viewer's emotions and adjust the character designation method based on the estimated viewer's emotions. For example, if the viewer is excited, the designation reception unit provides an interface that allows the viewer to quickly designate a character. If the viewer is relaxed, the designation reception unit displays detailed character information and increases the number of options. If the viewer is stressed, the designation reception unit provides a simple interface and minimizes the character designation procedure. This allows the character designation method to be adjusted according to the viewer's emotions, making it possible to designate a more appropriate character.

[0135] The process flow of the second embodiment will be briefly described below.

[0136] Step 1: The specification reception unit receives the specification of a character in the content by the viewer. The viewer can specify a character by clicking, using a voice command, using a gesture, or by other methods. The specification reception unit receives the viewer's input and transmits the specified character to the dialogue unit. Step 2: The dialogue unit executes a dialogue with the viewer using the character whose designation has been accepted by the designation acceptance unit. The dialogue unit recognizes the viewer's voice input and generates a voice output for the character. For example, when the viewer says "Hello," the character responds with "Hello, what's the matter?" The dialogue unit also recognizes the viewer's facial expressions and gestures, and changes the character's facial expressions and actions based on those. Furthermore, the dialogue unit estimates emotions from the viewer's voice and text, and adjusts the content of the dialogue based on the estimated emotions. Step 3: The selection reception unit receives the story selection from the viewer. The viewer can choose the development of the story from the presented options. For example, if the viewer selects "Next", the story will proceed to the next scene. Step 4: The providing unit provides the content of the story whose selection has been accepted by the selection accepting unit. The providing unit provides the content to the viewer based on the selected story. For example, when the viewer selects "Go on an adventure", the providing unit plays an adventure scene.

[0137] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0138] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0139] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0140] For example, each of the multiple elements including the designation reception unit, the dialogue unit, the selection reception unit, the provision unit, the suggestion unit, the presentation unit, and the emotion estimation function is realized by at least one of the smart device 14 and the data processing device 12. For example, the designation reception unit is realized by the control unit 46A of the smart device 14 and receives the viewer's input. The dialogue unit is realized by the specific processing unit 290 of the data processing device 12 and recognizes the viewer's voice input and generates the character's voice output. The selection reception unit is realized by the control unit 46A of the smart device 14 and receives the viewer's story selection. The provision unit is realized by the specific processing unit 290 of the data processing device 12 and provides the content of the selected story. The suggestion unit is realized by the specific processing unit 290 of the data processing device 12 and predicts the next story development based on the viewer's past selection. The presentation unit is realized by the control unit 46A of the smart device 14 and presents story options based on the historical background in which the viewer is interested. The emotion estimation function is realized by the specific processing unit 290 of the data processing device 12 and estimates the viewer's emotion and adjusts the dialogue content. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0141] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0142] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0143] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0144] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0145] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0146] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0147] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0148] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0149] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0150] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0151] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0152] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0153] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0154] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0155] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0156] For example, each of the multiple elements including the designation reception unit, the dialogue unit, the selection reception unit, the provision unit, the suggestion unit, the presentation unit, and the emotion estimation function is realized by at least one of the smart glasses 214 and the data processing device 12. For example, the designation reception unit is realized by the control unit 46A of the smart glasses 214 and receives the viewer's input. The dialogue unit is realized by the specific processing unit 290 of the data processing device 12 and recognizes the viewer's voice input and generates the character's voice output. The selection reception unit is realized by the control unit 46A of the smart glasses 214 and receives the viewer's story selection. The provision unit is realized by the specific processing unit 290 of the data processing device 12 and provides the content of the selected story. The suggestion unit is realized by the specific processing unit 290 of the data processing device 12 and predicts the next story development based on the viewer's past selection. The presentation unit is realized by the control unit 46A of the smart glasses 214 and presents story options based on the historical background in which the viewer is interested. The emotion estimation function is realized by the specific processing unit 290 of the data processing device 12, and estimates the emotion of the viewer and adjusts the dialogue content. The correspondence between each unit and the device or control unit is not limited to the above example, and various modifications are possible.

[0157] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0158] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0159] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0160] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0161] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0162] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0163] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0164] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0165] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0166] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0167] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0168] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0169] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0170] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0171] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0172] For example, each of the multiple elements including the designation reception unit, the dialogue unit, the selection reception unit, the provision unit, the suggestion unit, the presentation unit, and the emotion estimation function is realized by at least one of the headset type terminal 314 and the data processing device 12. For example, the designation reception unit is realized by the control unit 46A of the headset type terminal 314 and receives the viewer's input. The dialogue unit is realized by the specific processing unit 290 of the data processing device 12 and recognizes the viewer's voice input and generates the character's voice output. The selection reception unit is realized by the control unit 46A of the headset type terminal 314 and receives the viewer's story selection. The provision unit is realized by the specific processing unit 290 of the data processing device 12 and provides the content of the selected story. The suggestion unit is realized by the specific processing unit 290 of the data processing device 12 and predicts the next story development based on the viewer's past selection. The presentation unit is realized by the control unit 46A of the headset type terminal 314 and presents story options based on the historical background in which the viewer is interested. The emotion estimation function is realized by the specific processing unit 290 of the data processing device 12, and estimates the emotion of the viewer and adjusts the dialogue content. The correspondence between each unit and the device or control unit is not limited to the above example, and various modifications are possible.

[0173] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0174] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0175] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0176] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0177] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0178] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0179] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0180] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0181] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0182] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0183] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0184] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0185] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0186] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0187] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0188] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0189] For example, each of the multiple elements including the designation receiving unit, the dialogue unit, the selection receiving unit, the provision unit, the suggestion unit, the presentation unit, and the emotion estimation function is realized by at least one of the robot 414 and the data processing device 12. For example, the designation receiving unit is realized by the control unit 46A of the robot 414 and receives the viewer's input. The dialogue unit is realized by the specific processing unit 290 of the data processing device 12 and recognizes the viewer's voice input and generates the character's voice output. The selection receiving unit is realized by the control unit 46A of the robot 414 and receives the viewer's story selection. The provision unit is realized by the specific processing unit 290 of the data processing device 12 and provides the content of the selected story. The suggestion unit is realized by the specific processing unit 290 of the data processing device 12 and predicts the next story development based on the viewer's past selection. The presentation unit is realized by the control unit 46A of the robot 414 and presents story options based on the historical background in which the viewer is interested. The emotion estimation function is realized by the specific processing unit 290 of the data processing device 12 and estimates the viewer's emotion and adjusts the dialogue content. The correspondence between each part and the device or control part is not limited to the above-mentioned example, and various modifications are possible.

[0190] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0191] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0192] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0193] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out on emotion map 400 you go, the more visible the emotions become (the more they are expressed in behavior).

[0194] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0195] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0196] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0197] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0198] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0199] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0200] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0201] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0202] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0203] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0204] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0205] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0206] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0207] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0208] (Appendix 1) a designation receiving unit that receives designation of a character in the content by a viewer; a dialogue unit that executes a dialogue between the character designated by the designation reception unit and the viewer; a selection receiving unit that receives a selection of a story from the viewer; a providing unit that provides the content as content of a story selected by the selection receiving unit; A system characterized by: (Appendix 2) The dialogue unit includes: Recognize the viewer's voice input and generate voice output for the character 2. The system of claim 1, (Appendix 3) The dialogue unit includes: Recognizes the viewer's facial expressions and changes the character's facial expressions based on the viewer's 2. The system of claim 1, (Appendix 4) The dialogue unit includes: Recognize the viewer's gestures and control the character's movements 2. The system of claim 1, (Appendix 5) The dialogue unit includes: Infer emotions from the viewer's voice and text, and adjust dialogue content based on the inferred emotions 2. The system of claim 1, (Appendix 6) The dialogue unit includes: Equipped with a feature that allows viewers to choose the voice actors of the characters 2. The system of claim 1, (Appendix 7) Equipped with a suggestion section that predicts the next story development based on the viewer's past choices and suggests the predicted story 2. The system of claim 1, (Appendix 8) It has a presentation section that presents story options based on historical background that interests the viewer. 2. The system of claim 1, (Appendix 9) The dialogue unit includes: Execute dialogue between multiple characters selected by the viewer 2. The system of claim 1, (Appendix 10) The designation reception unit is Estimate the viewer's emotions and adjust the character assignment method based on the estimated viewer's emotions 2. The system of claim 1, (Appendix 11) The designation reception unit is Analyze the viewer's past character selection history and select the optimal selection method 2. The system of claim 1, (Appendix 12) The designation reception unit is When specifying a character, filter based on the viewer's current interests 2. The system of claim 1, (Appendix 13) The designation reception unit is Estimate the viewer's emotions and determine the priority of characters to be assigned based on the estimated viewer's emotions. 2. The system of claim 1, (Appendix 14) The designation reception unit is When selecting characters, prioritize characters that are more relevant to the viewer based on their geographic location. 2. The system of claim 1, (Appendix 15) The designation reception unit is When a character is selected, analyze the viewer's social media activity and select a related character. 2. The system of claim 1, (Appendix 16) The dialogue unit includes: Estimate the viewer's emotions and adjust the dialogue expression based on the estimated viewer's emotions 2. The system of claim 1, (Appendix 17) The dialogue unit includes: When interacting with a viewer, the system generates optimal dialogue content by referencing the viewer's past dialogue history. 2. The system of claim 1, (Appendix 18) The dialogue unit includes: As you interact, tailor the conversation to your audience based on their current interests 2. The system of claim 1, (Appendix 19) The dialogue unit includes: Estimate viewer sentiment and adjust dialogue length based on the estimated viewer sentiment 2. The system of claim 1, (Appendix 20) The dialogue unit includes: During dialogue, the optimal dialogue content is generated taking into account the viewer's geographic location information 2. The system of claim 1, (Appendix 21) The dialogue unit includes: During interactions, analyze your audience's social media activity and generate relevant conversation content 2. The system of claim 1, (Appendix 22) The selection receiving unit is Inferring viewer sentiment and adjusting selection acceptance based on the estimated viewer sentiment 2. The system of claim 1, (Appendix 23) The selection receiving unit is Analyze the viewer's past selection history and select the optimal selection method 2. The system of claim 1, (Appendix 24) The selection receiving unit is Filter story selection based on your audience's current interests 2. The system of claim 1, (Appendix 25) The selection receiving unit is Estimate audience sentiment and prioritize story selection based on estimated audience sentiment 2. The system of claim 1, (Appendix 26) The selection receiving unit is When selecting stories, take into account the viewer's geographic location to prioritize the most relevant stories 2. The system of claim 1, (Appendix 27) The selection receiving unit is When selecting stories, analyze your audience's social media activity and choose relevant stories 2. The system of claim 1, (Appendix 28) The providing unit is Estimate viewer sentiment and adjust content delivery based on the estimated viewer sentiment 2. The system of claim 1, (Appendix 29) The providing unit is Analyze viewers' past content viewing history and select the optimal delivery method 2. The system of claim 1, (Appendix 30) The providing unit is Filter content as it is served based on the viewer's current interests 2. The system of claim 1, (Appendix 31) The providing unit is Estimate viewer sentiment and prioritize content based on the estimated viewer sentiment 2. The system of claim 1, (Appendix 32) The providing unit is When serving content, it takes into account the viewer's geographic location to prioritize the most relevant content. 2. The system of claim 1, (Appendix 33) The providing unit is When serving content, analyze your social media activity and provide you with relevant content 2. The system of claim 1, (Appendix 34) The suggestion unit, Inferring viewer sentiment and adjusting the presentation of suggestions based on the estimated viewer sentiment 2. The system of claim 1, (Appendix 35) The suggestion unit, When making suggestions, the system generates optimal suggestions by referring to the viewer's past selection history 2. The system of claim 1, (Appendix 36) The suggestion unit, When making a suggestion, tailor the suggestion based on the viewer's current interests 2. The system of claim 1, (Appendix 37) The suggestion unit, Estimate viewer sentiment and adjust the length of suggestions based on the estimated viewer sentiment 2. The system of claim 1, (Appendix 38) The suggestion unit, When making suggestions, the geographic location of the viewer is taken into account to generate optimal suggestions. 2. The system of claim 1, (Appendix 39) The suggestion unit, When making a recommendation, analyze your audience's social media activity to generate relevant recommendations 2. The system of claim 1, (Appendix 40) The presentation unit is Estimating viewer emotions and adjusting presentation based on the estimated viewer emotions 2. The system of claim 1, (Appendix 41) The presentation unit is When presenting content, the system generates optimal content by referring to the viewer's past selection history. 2. The system of claim 1, (Appendix 42) The presentation unit is At the time of presentation, customize the presentation based on the viewer's current areas of interest 2. The system of claim 1, (Appendix 43) The presentation unit is Estimating viewer emotion and adjusting presentation length based on the estimated viewer emotion 2. The system of claim 1, (Appendix 44) The presentation unit is At the time of presentation, the viewer's geographic location is taken into account to generate the optimal presentation content 2. The system of claim 1, (Appendix 45) The presentation unit is At the time of presentation, analyze the viewer's social media activity and generate relevant presentation content 2. The system of claim 1, [Explanation of symbols]

[0209] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A designation receiving unit that receives designation of a character in the content by a viewer; a dialogue unit that executes a dialogue between the character designated by the designation reception unit and the viewer; a selection receiving unit that receives a selection of a story from the viewer; a providing unit that provides the content as content of a story selected by the selection receiving unit; A system characterized by:

2. The dialogue unit includes: Recognizing voice input from said viewer and generating voice output for said character 2. The system of claim 1 .

3. The dialogue unit includes: Recognizing the facial expression of the viewer and changing the character's facial expression based on the facial expression of the viewer 2. The system of claim 1 .

4. The dialogue unit includes: Recognizing the gestures of the viewer and controlling the character's movements 2. The system of claim 1 .

5. The dialogue unit includes: Estimating emotions from the voice and text of the viewer and adjusting the dialogue content based on the estimated emotions 2. The system of claim 1 .

6. The dialogue unit includes: The viewer has the ability to select the voice actor for the character 2. The system of claim 1 .

7. A suggestion unit that predicts the next story development based on the viewer's past selections and suggests the predicted story.

2. The system of claim 1 .

8. A presentation unit that presents story options based on a historical background that is of interest to the viewer.

2. The system of claim 1 .

Images